UNESCO module: Introduction to Computer Vision and Image Processing Department of Pattern Recognition and Knowledge Engineering Institute of Information Technology Hanoi, Vietnam Represented by LUONG CHI MAI lcmai@ioit.ncst.ac.vn Outline of the presentation Objectives, Brief Discussion Prerequisite Introduction and and Content to Lectures Conclusion This presentation summarizes the content and organization of lectures in module Image Processing and Computer Vision. Objectives The course provides fundamental techniques of Image Processing and Computer Vision as well issues in practical use. Prerequisite ✦ A basic background in mathematics and computers is necessary, ✦ Knowledge of the C programming language will enhance the usefulness of the algorithms used in programming, ✦ Understanding of signal and system theory is helpful in mastering transforms and compression. Target audience ✦ ✦ Engineers, programmers, graphics specialists, multimedia developers, and imaging professionals will all appreciate Computer Vision and Image Processing's solid introduction Anyone who uses computer imaging. What’s the Image Processing? Image Processing (IP) is used for two somewhat different purposes: a. improving the visual appearance of images to a human view, and b. preparing images for measurement of the features and structures present. Image Processing:= Image → Image Transformation ➊ What’s Computer Vision ? ❷ Computer Vision (CV): to create a model of the real word from images. A CV system recovers useful information about a scene from its two-dimensional projections. This recover requires the inversion of a many-toone mapping. Vision:=Geometry+Measurement+Interpretatio n Relationships between subjects (1) Many fields are related to Computer Vision Image Processing (IP): techniques usually transform images into other images, (enhancement, correcting blurred, out-of-focus, compression → better 2D projection image for CV).The task of information recovery is left to human user. Computer Graphics (CG): generates images from geometric primitives such as lines, circles, and free-form surfaces. CV is the inverse problem: estimating the geometric primitives and other features from images. CG: Synthesis of images. CV: Analysis of images. Relationships between subjects (2) Pattern Recognition (PR): classifies numerical and symbolic data. Techniques: statistical and syntactical. PR techniques play an important role in CV for recognizing objects. Object recognition in CV usually requires many other techniques. Artificial Intelligence (AI): is concerned with designing systems that are intelligent and with studying computational aspects of intelligent. CV is often considered as a sub-field of AI Psyochophysics: along with cognitive science, studies human vision for a long time. Many techniques in CV are related to what is known abut human vision. Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression About the Chapters Chapters ✦ 1, 2, 3, 4, 5, 9, 10 related to Image Processing: well known techniques to enhancement images. ✦ 6, 7, 8 related to Computer Visions Image presentation (1) 1.1 Image capture, representation, and storage: digital image, DPI, pixel... Example: Variouse quantizing level: (a) 6 bits; (b) 4 bits; (c) 2 bits; (d) 1 bit. Image presentation (2) ✦ 1.2 Color representation: Color systems: RGB, CMY/CMYK, HSI, YCbCr Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Statistical operations (1) The algorithms are independent of the position of the pixels. Basic operation: Histogram transformation 2.1 Gray-level transformation - Intensity transformation - Look-up-table techniques - Gamma correction function - Contrast streching End-in-search 2.2 Histogram equalization Statistical operations (2) 2.3 Multi-image operations –Background substraction –Multi-image averaging New-Pixel = α Pixel1 + (1 - α )Pixel2 Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Spatial operations and transformations (1) Combining the techniques and operations that deal with pixels and their neighbors (spatial operations). - Spatial filters (normally removing noise by reference to the neighboring pixel values), - Weighted averaging of pixel areas (convolutions), - Comparing areas on an image with known pixel area shapes so as to find shapes in images (correlation). - Edge detection and on detection of "interest point". Spatial operations and transformations (2) Basic operation: Templates and Convolution n −1 m −1 T ∗ I ( X , Y ) = ∑∑ T (i, j ) I ( X + i, Y + j ) i =0 j =0 I(x,y) - image T(i,j) - template of the size n x m Spatial operations and transformations (3) ✦ 3.3 Other window operations – Median filtering – k-closest averaging – Interest point – Moravec operator – Correlation Spatial operations and transformations (4) ✦3.4 Two dimensional geometric transformations Frequently it is useful to zoom in on a part of an image, rotate, shift, skew or zoom out from an image. If (x’,y’) - the new coordinates and (x, y) - original coordinates – Forward Transformation (x’,y’) = f(x, y) for all (x, y) is created. – Invest Transformation I(x, y) = F(old image, x’, y’) Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Segmentation and edge detection (1) Segmentation: basic requirement for the identification and classification of objects in scene. ✦ Techniques: splitting an image up into segments (also call regions or areas), each holds some property distinct from their neighbor. ✦ ✦ Approaches : - identifying the edges (or lines) that run through an image - identifying regions (or areas) within an image. Region operations is the dual of edge operations. Ideally edge and region operations should give the same segmentation result, however, in practice the two rarely correspond. Segmentation and edge detection (2) ✦ 4.1 Region operations – Crudge edge detection – Region merging – Region spliting ✦ 4.2 Basic edge detection Segmentation and edge detection (3) ✦ 4.3 First order derivative for edge detection Hc = y_differ(x, y) = value(x, y) – value(x, y+1) Hr = X_differ(x, y) = value(x, y) – value(x-1, y) ✦ 4.3 Second-order edge detection ✦ 4.4 Pyramid edge detection ✦ 4.5 Crack edge detection ✦ 4.6 Edge following Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Morphological and other area operations (1) Morphological defined - Morphology means the form and structure of an object, it’s related to shape - Digital morphology is a way to describe or analyze the shape of a digital object. Morphological operations (2) 5.2 Basic morphological operations – Binary dilation – Binary erosion ✦5.3 Opening and closing operators Example: The use of opening: (a) An image having many connected objects, (b) Objects can be isolated by opening using the simple structuring element, (c) An image that has been subjected to noise, (d) The noisy image after opening showing that the black noise pixels have been removed. Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Finding basic shapes (1) ✦ ✦ Previous chapters dealt with purely statistical and spatial operations. Techniques: - looking at and processing whole images - uses information generated by the algorithms in the previous chapter. - finding basic two-dimensional shapes or elements of shapes by putting edges together to form lines that are likely represent real edges. Finding basic shapes (2) y ✦ 6.2 Hough transforms ✦ 6.3 Bresenham’s algorithms ✦ 6.4 Using interest point ✦ 6.5 Labeling lines and regions (x,y) r θ Shotest distance from origin to line defines the line in term of r and θ x One of many possible lines through (x,y), e.g. y=ax+b Four cicles coincide here only Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Reasoning, facts and inference (1) - Moving from the standard IP approach to CV to make statement about the geometry of objects and allocate labels to them. - Enhancing by making reasoned statements, by codifying facts, and making judgments based on past experience. - Introducing to some concepts in logical reasoning that relate specifically to CV. - Introducing training aspects of reasoning systems. The reasoning is the highest level of CV processing. Reasoning, facts and inference (2) ✦ 7.1 Facts and Rules - Constructing a set of facts - Constructing a rule base. ✦7.2 Strategic learning Example: A pedestal training and a pedestal description Reasoning, facts and inference (3) ✦7.3 Networks and spatial descriptors Example: Elementary network of spatial relationship – – – – L is all element of C is a subset of P with the visual property or R at this position with respect to ✦7.4 Rule orders C Table L Top Legs Shyni P R L Leg Above R Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Pattern recognition and training (1) ✦ ✦ ✦ Previous chapter presented some methods used in reasoning about facts from image: edges or textures, colours or surface positions. Some problems are better described as problems of determining a high level fact from a pattern of some kind. The term "pattern" has a wide range of meanings, We are particularly interested in sets of value that describe things, normally where the set of values is of a known size. This is different to looking at a scene of a flat surfaced object where we do not know how many corners there are, how many edges or how many surfaces. Pattern recognition and training (2) ✦ 8.1 General problem Image Make a series of measurements to give a set of values x1 xn Determine which object this set of measurements suggests is in the image O1 On M A X M U M Decision making process Decision function generator Pattern vector Score vector (highest object score is choosen) object =... Pattern recognition and training (3) ✦ 8.2 Approaches to the decision making process ✦ 8.3 Decision functions ✦ 8.4 Determining decision functions ✦ 8.5 Non-linear decision functions ✦ 8.6 Using cluster means ✦ 8.7 Supervised and unsupervised learning - Statistical: Bayesian likelihood supervised learning - Syntactical learning. Pattern recognition and training (4) ✦ 8.4 Determining decision function: - Searching for islands of simplicity, - Distance or similarity measure, C A Group A B Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression The frequency domain (1) Most signal processing is done in a mathematical space known as the frequency domain. ✦ In order to represent data in the frequency domain, some transform is necessary. ✦ The signal frequency of an image refers to the rate at which the pixel intensities change. ✦ - The high frequencies are concentrated around the axes dividing the image into quadrants. - The corners have lower frequencies. Low spatial frequencies are noted by large areas of nearly constant values. The frequency domain (2) Fourier Transform of a spot: (a) original image; (b) Fourier Transform. ✦ 9.1 The Harley transform ✦ 9.2 The Fourier transform Content of the course Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Image Compression (1) Compression of images: problem of storing them in a form that systems need to get the following benefits: ✦ - speedily operation (both compression and unpacking), - significant reduction in required memory, no significant loss of quality in the image, - format of output suitable for transfer or storage. Each of this depends on the user and the application. Image Compression (2) A typical data compression system. Image Compression (3) ✦ Run Length Encoding ✦ Huffman Coding ✦ Modified Huffman Coding ✦ Modified READ ✦ Arithmetic Coding ✦ LZW ✦ JPEG ✦ Other state-of-the-art image compression methods: Fractal and Wavelet compression. Conclusion Improvement ✦ Focus to recovering from 2D projection to create a object model: - Coordinate system and camera calibration - Curve and surfaces - Dynamic vision ✦ Object recognition [...]...About the Chapters Chapters ✦ 1, 2, 3, 4, 5, 9, 10 related to Image Processing: well known techniques to enhancement images ✦ 6, 7, 8 related to Computer Visions Image presentation (1) 1.1 Image capture, representation, and storage: digital image, DPI, pixel Example: Variouse quantizing level: (a) 6 bits; (b) 4 bits; (c) 2 bits; (d) 1 bit Image presentation (2) ✦ 1.2 Color representation: Color systems:... Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Reasoning, facts and inference (1) - Moving from the standard IP approach to CV to make statement... operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Segmentation and edge detection (1) Segmentation: basic requirement for the identification and classification of objects in scene ✦ Techniques: splitting an image. .. 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Morphological and other area operations (1) Morphological defined - Morphology means the form and. .. Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Spatial operations and transformations (1) Combining the techniques and operations... facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Finding basic shapes (1) ✦ ✦ Previous chapters dealt with purely statistical and spatial operations Techniques: - looking at and processing whole images - uses information generated by the algorithms in the previous chapter - finding basic two-dimensional shapes or elements of shapes by putting edges together... geometry of objects and allocate labels to them - Enhancing by making reasoned statements, by codifying facts, and making judgments based on past experience - Introducing to some concepts in logical reasoning that relate specifically to CV - Introducing training aspects of reasoning systems The reasoning is the highest level of CV processing Reasoning, facts and inference (2) ✦ 7.1 Facts and Rules - Constructing... that deal with pixels and their neighbors (spatial operations) - Spatial filters (normally removing noise by reference to the neighboring pixel values), - Weighted averaging of pixel areas (convolutions), - Comparing areas on an image with known pixel area shapes so as to find shapes in images (correlation) - Edge detection and on detection of "interest point" Spatial operations and transformations... Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression Statistical operations (1) The algorithms... Chapter Chapter Chapter Chapter Chapter Chapter Chapter 1: Image presentation 2: Statistic operations 3: Spatial operations and transformations 4: Segmentation and edge detection 5: Morphological and other area area operations 6: Finding basic shapes 7: Reasoning, facts and inference 8: Pattern recognition and training 9: Frequency domain 10: Image compression