TeAM YYePG Digitally signed by TeAM YYePG DN: cn=TeAM YYePG, c=US, o=TeAM YYePG, ou=TeAM YYePG, email=yyepg@msn.com Reason: I attest to the accuracy and integrity of this document Date: 2005.04.11 14:42:21 +08'00' Video Data Management and Information Retrieval Sagarmay Deb University of Southern Queensland, Australia IRM Press Publisher of innovative scholarly and professional information technology titles in the cyberage Hershey • London • Melbourne • Singapore Acquisitions Editor: Mehdi Khosrow-Pour Senior Managing Editor: Jan Travers Managing Editor: Amanda Appicello Development Editor: Michele Rossi Copy Editor: Jane Conley Typesetter: Jennifer Wetzel Cover Design: Lisa Tosheff Printed at: Yurchak Printing Inc. Published in the United States of America by IRM Press (an imprint of Idea Group Inc.) 701 E. Chocolate Avenue, Suite 200 Hershey PA 17033-1240 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: cust@idea-group.com Web site: http://www.irm-press.com and in the United Kingdom by IRM Press (an imprint of Idea Group Inc.) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 3313 Web site: http://www.eurospan.co.uk Copyright © 2005 by IRM Press. All rights reserved. No part of this book may be reproduced in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Library of Congress Cataloging-in-Publication Data Video data management and information retrieval / Sagarmay Deb, editor. p. cm. Includes bibliographical references and index. ISBN 1-59140-571-8 (h/c) ISBN 1-59140-546-7 (s/c) ISBN 1-59140-547-5 (ebook) 1. Digital video. 2. Database management. 3. Information storage and retrieval systems. I. Deb, Sagarmay, 1953- TK6680.5.V555 2004 006 dc22 2004022152 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. Video Data Management and Information Retrieval Table of Contents Preface vi Sagarmay Deb, University of Southern Queensland, Australia S ECTION I: AN INTRODUCTION TO VIDEO DATA MANAGEMENT AND INFORMATION RETRIEVAL Chapter I Video Data Management and Information Retrieval 1 Sagarmay Deb, University of Southern Queensland, Australia S ECTION II: V IDEO DATA STORAGE TECHNIQUES AND NETWORKING Chapter II HYDRA: High-performance Data Recording Architecture for Streaming Media 9 Roger Zimmermann, University of Southern California, USA Kun Fu, University of Southern California, USA Dwipal A. Desai, University of Southern California, USA Chapter III Wearable and Ubiquitous Video Data Management for Computational Augmentation of Human Memory 33 Tatsuyuki Kawamura, Nara Institute of Science and Technology, Japan Takahiro Ueoka, Nara Institute of Science and Technology, Japan Yasuyuki Kono, Nara Institute of Science and Technology, Japan Masatsugu Kidode, Nara Institute of Science and Technology, Japan Chapter IV Adaptive Summarization of Digital Video Data 77 Waleed E. Farag, Zagazig University, Egypt Hussein Abdel-Wahab, Old Dominion University, USA Chapter V Very Low Bit-rate Video Coding 100 Manoranjan Paul, Monash University, Australia Manzur Murshed, Monash University, Australia Laurence S. Dooley, Monash University, Australia S ECTION III: VIDEO DATA SECURITY AND VIDEO DATA SYNCHRONIZATION AND TIMELINESS Chapter VI Video Biometrics 149 Mayank Vatsa, Indian Institute of Technology, Kanpur, India Richa Singh, Indian Institute of Technology, Kanpur, India P. Gupta, Indian Institute of Technology, Kanpur, India Chapter VII Video Presentation Model 177 Hun-Hui Hsu, Tamkang University, Taiwan, ROC Yi-Chun Liao, Tamkang University, Taiwan, ROC Yi-Jen Liu, Tamkang University, Taiwan, ROC Timothy K. Shih, Tamkang University, Taiwan, ROC SECTION IV: VIDEO SHOT BOUNDARY DETECTION Chapter VIII Video Shot Boundary Detection 193 Waleed E. Farag, Zagazig University, Egypt Hussein Abdel-Wahab, Old Dominion University, USA Chapter IX Innovative Shot Boundary Detection for Video Indexing 217 Shu-Ching Chen, Florida International University, USA Mei-Ling Shyu, University of Miami, USA Chengcui Zhang, University of Alabama at Birmingham, USA Chapter X A Soft-Decision Histogram from the HSV Color Space for Video Shot Detection 237 Shamik Sural, Indian Institute of Technology, Kharagpur, India M. Mohan, Indian Institute of Technology, Kharagpur, India A.K. Majumdar, Indian Institute of Technology, Kharagpur, India SECTION V: VIDEO FEATURE EXTRACTIONS Chapter XI News Video Indexing and Abstraction by Specific Visual Cues: MSC and News Caption 254 Fan Jiang, Tsinghua University, China Yu-Jin Zhang, Tsinghua University, China S ECTION VI: V IDEO INFORMATION RETRIEVAL Chapter XII An Overview of Video Information Retrieval Techniques 283 Sagarmay Deb, University of Southern Queensland, Australia Yanchun Zhang, Victoria University of Technology, Australia Chapter XIII A Framework for Indexing Personal Videoconference 293 Jiqiang Song, Chinese University of Hong Kong, Hong Kong Michael R. Lyu, Chinese University of Hong Kong, Hong Kong Jenq-Neng Hwang, University of Washington, USA Chapter XIV Video Abstraction 321 Jung Hwan Oh, The University of Texas at Arlington, USA Quan Wen, The University of Texas at Arlington, USA Sae Hwang, The University of Texas at Arlington, USA Jeongkyu Lee, The University of Texas at Arlington, USA Chapter XV Video Summarization Based on Human Face Detection and Recognition 347 Hong-Mo Je, Pohang University of Science and Technology, Korea Daijin Kim, Pohang University of Science and Technology, Korea Sung-Yang Bang, Pohang University of Science and Technology, Korea About the Authors 379 Index 389 Preface vi INTRODUCTION Video data management and information retrieval are very important areas of research in computer technology. Plenty of research is being done in these fields at present. These two areas are changing our lifestyles because together they cover creation, maintenance, accessing, and retrieval of video, audio, speech, and text data and information for video display. But still lots of important issues in these areas remain unresolved and further research is needed to be done for better techniques and appli- cations. The primary objective of the book is to combine these two related areas of re- search together and provide an up-to-date account of the work being done. We ad- dressed research issues in those fields where some progress has already been made. Also, we encouraged researchers, academics, and industrial technologists to provide new and brilliant ideas on these fields that could be pursued for further research. Section I gives an introduction. We have given general introduction of the two areas, namely, video data management and information retrieval, from the very elemen- tary level. We discussed the problems in these areas and some of the work done in these fields since the last decade. Section II defines video data storage techniques and networking. We present a chapter that describes the design for a High-performance Data Recording Architecture (HYDRA) that can record data in real time for large-scale servers. Although digital continuous media (CM) is being used as an integral part of many applications and attempts have been made for efficient retrieval of such media for many concurrent users, not much has been done so far to implement these ideas for large-scale servers. Then a chapter introduces video data management techniques for computational aug- mentation of human memory, i.e., augmented memory, on wearable and ubiquitous com- puters used in our everyday life. In another chapter, in order to organize and manipulate vast amount of multimedia data in an efficient way, a method to summarize these digital data has been presented. Also we present a contemporary review of the various differ- ent strategies available to facilitate Very Low Bit-Rate (VLBR) coding for video commu- nications over mobile and fixed transmission channels as well as the Internet. vii Section III talks about video data security and video data synchronization and timeliness. We describe how to present different multimedia objects on a web-based presentation system. A chapter is devoted to highlighting the biometrics technologies, which are based on video sequences, viz face, eye (iris/retina), and gait. Section IV will present various video shot boundary detection techniques. A new robust paradigm capable of detecting scene changes on compressed MPEG video data directly has been proposed. Then an innovative shot boundary detection method us- ing an unsupervised segmentation algorithm and the technique of object tracking based on the segmentation mask maps are presented. We also describe a histogram with soft decision using the Hue, Saturation, and Intensity (HSV) color space for effective detec- tion of video shot boundaries. Section V will throw light on video feature extractions. We address the issues of providing the semantic structure and generating abstraction of content in news broad- cast. Section VI covers video information retrieval techniques and presents an up-to- date overview of various video information retrieval systems. As the rapid technical advances of multimedia communication have made it possible for more and more people to enjoy videoconferences, important issues unique to personal videoconference and a comprehensive framework for indexing personal videoconference have been presented. Then we have dealt with video summarization using human facial information through face detection and recognition and also a discussion on various issues of video ab- straction with a new approach to generate it. The audience for this book would be researchers who are working in these two fields. Also researchers from other areas who could start-up in these fields could find the book useful. It could be a reference guide for researchers from other related areas as well. Reading this book can benefit undergraduate and post-graduate students who are interested in multimedia and video technology. CHAPTER HIGHLIGHTS In Chapter I, Video Data Management and Information Retrieval, we present a basic introduction of the two very important areas of research in the domain of Informa- tion Technology, namely, video data management and video information retrieval. Both of these areas still need research efforts to seek solutions to many unresolved prob- lems for efficient data management and information retrieval. We discuss those issues and relevant work done in these two fields during the last few years. Chapter II, HYDRA: High-performance Data Recording Architecture for Stream- ing Media, describes the design for a High-performance Data Recording Architecture (HYDRA). Presently, digital continuous media (CM) are well established as an integral part of many applications. In recent years, a considerable amount of research has focused on the efficient retrieval of such media for many concurrent users. The authors argue that scant attention has been paid to large-scale servers that can record such streams in real time. However, more and more devices produce direct digital output streams either over wired or wireless networks, and various applications are emerging to make use of them. For example, in many industrial applications, cameras now provide the means to monitor, visualize, and diagnose events. Hence, the need arises to capture and store these streams with an efficient data stream recorder that can handle both viii recording and playback of many streams simultaneously and provide a central reposi- tory for all data. With this chapter, the authors present the design of the HYDRA system, which uses a unified architecture that integrates multi-stream recording and retrieval in a coherent paradigm, and hence provides support for these emerging appli- cations. Chapter III, Wearable and Ubiquitous Video Data Management for Computa- tional Augmentation of Human Memory, introduces video data management techniques for computational augmentation of human memory, i.e., augmented memory, on wear- able and ubiquitous computers used in our everyday life. The ultimate goal of aug- mented memory is to enable users to conduct themselves using human memories and multimedia data seamlessly anywhere, anytime. In particular, a user’s viewpoint video is one of the most important triggers for recalling past events that have been experi- enced. We believe designing an augmented memory system is a practical issue for real- world video data management. This chapter also describes a framework for an aug- mented memory album system named Scene Augmented Remembrance Album (SARA). In the SARA framework, we have developed three modules for retrieving, editing, trans- porting, and exchanging augmented memory. Both the Residual Memory module and the I’m Here! module enable a wearer to retrieve video data that he/she wants to recall in the real world. The Ubiquitous Memories module is proposed for editing, transport- ing, and exchanging video data via real-world objects. Lastly, we discuss future works for the proposed framework and modules. Chapter IV is titled Adaptive Summarization of Digital Video Data. As multime- dia applications are rapidly spread at an ever-increasing rate, efficient and effective methodologies for organizing and manipulating these data become a necessity. One of the basic problems that such systems encounter is to find efficient ways to summarize the huge amount of data involved. In this chapter, we start by defining the problem of key frames extraction then reviewing a number of proposed techniques to accomplish that task, showing their pros and cons. After that, we describe two adaptive algorithms proposed in order to effectively select key frames from segmented video shots where both apply a two-level adaptation mechanism. These algorithms constitute the second stage of a Video Content-based Retrieval (VCR) system that has been designed at Old Dominion University. The first adaptation level is based on the size of the input video file, while the second level is performed on a shot-by-shot basis in order to account for the fact that different shots have different levels of activity. Experimental results show the efficiency and robustness of the proposed algorithms in selecting the near optimal set of key frames required, to represent each shot. Chapter V, Very Low Bit-rate Video Coding, presents a contemporary review of the various different strategies available to facilitate Very Low Bit Rate (VLBR) coding for video communications over mobile and fixed transmission channels and the Internet. VLBR media is typically classified as having a bit rate between 8 and 64Kbps. Tech- niques that are analyzed include Vector Quantization, various parametric model-based representations, the Discrete Wavelet and Cosine Transforms, and fixed and arbitrary shaped pattern-based coding. In addition to discussing the underlying theoretical prin- ciples and relevant features of each approach, the chapter also examines their benefits and disadvantages together with some of the major challenges that remain to be solved. The chapter concludes by providing some judgments on the likely focus of future research in the VLBR coding field. ix Chapter VI is titled Video Biometrics. Biometrics is a technology of fast, user- friendly personal identification with a high level of accuracy. This chapter highlights the biometrics technologies that are based on video sequences viz face, eye (iris/ retina), and gait. The basics behind the three video-based biometrics technologies are discussed along with a brief survey. Chapter VII is titled Video Presentation Model. Lecture-on-Demand (LOD) multi- media presentation technologies among the network are most often used in many com- munications services. Examples of those applications include video-on- demand, inter- active TV, and the communication tools of a distance learning system, and so on. We describe how to present different multimedia objects on a web-based presentation system. Using characterization of extended media streaming technologies, we devel- oped a comprehensive system for advanced multimedia content production: support for recording the presentation, retrieving the content, summarizing the presentation, and customizing the representation. This approach significantly impacts and supports the multimedia presentation authoring processes in terms of methodology and commer- cial aspects. Using the browser with the Windows media services allows students to view live video of the teacher giving his speech, along with synchronized images of his presentation slides and all the annotations/comments. In our experience, this very approach is sufficient for use in a distance learning environment. Chapter VIII is titled Video Shot Boundary Detection. The increasing use of multimedia streams nowadays necessitates the development of efficient and effective methodologies for manipulating databases storing this information. Moreover, con- tent-based access to video data requires in its first stage to parse each video stream into its building blocks. The video stream consists of a number of shots; each one of them is a sequence of frames pictured using a single camera. Switching from one camera to another indicates the transition from a shot to the next one. Therefore, the detection of these transitions, known as scene change or shot boundary detection, is the first step in any video analysis system. A number of proposed techniques for solving the problem of shot boundary detection exist, but the major criticisms of them are their inefficiency and lack of reliability. The reliability of the scene change detection stage is a very significant requirement because it is the first stage in any video retrieval system; thus, its performance has a direct impact on the performance of all other stages. On the other hand, efficiency is also crucial due to the voluminous amounts of informa- tion found in video streams. This chapter proposes a new robust and efficient paradigm capable of detecting scene changes on compressed MPEG video data directly. This paradigm constitutes the first part of a Video Content-based Retrieval (VCR) system that has been designed at Old Dominion University. Initially, an abstract representation of the compressed video stream, known as the DC sequence, is extracted, then it is used as input to a Neural Network Module that performs the shot boundary detection task. We have studied experimentally the performance of the proposed paradigm and have achieved higher shot boundary detection and lower false alarms rates compared to other tech- niques. Moreover, the efficiency of the system outperforms other approaches by sev- eral times. In short, the experimental results show the superior efficiency and robust- ness of the proposed system in detecting shot boundaries and flashlights (sudden lighting variation due to camera flash occurrences) within video shots. Chapter IX is titled Innovative Shot Boundary Detection for Video Indexing. Recently, multimedia information, especially the video data, has been made overwhelm- [...]... Editor, and Carrie Skovrinskie, Office Manager Finally, the editor would like to thank his wife Ms Clera Deb for her support and cooperation during the venture Sagarmay Deb University of Southern Queensland, Australia Section I An Introduction to Video Data Management and Information Retrieval Video Data Management and Information Retrieval 1 Chapter I Video Data Management and Information Retrieval. .. Queensland, Australia ABSTRACT In this chapter, we present a basic introduction to two very important areas of research in the domain of Information Technology, namely, video data management and video information retrieval Both of these areas need additional research efforts to seek solutions to many unresolved problems for efficient data management and information retrieval We discuss those issues and. .. to catalog, index, and retrieve the stored video data, namely, video boundary detection, video database indexing, and video abstraction The topic of this chapter is Video Abstraction, which deals with short representation of an original video and helps to enable the fast browsing and retrieving of the representative contents A general view of video abstraction, its related works, and a new approach... issues briefly and ends with a summary RELATED ISSUES AND RELEVANT WORKS Video Data Management With the rapid advancement and development of multimedia technology during the last decade, the importance of managing video data efficiently has increased tremendously To organize and store video data in a standard way, vast amounts of data are being converted to digital form Because the volume of data is enormous,... Group Inc is prohibited Video Data Management and Information Retrieval 7 Manjunath, B.S., & Ma, W.Y (1996) Texture features for browsing and retrieval of image data IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(8) Marques, O., & Furht, B (2002) Content-based visual information retrieval In T.K Shih (Ed.), Distributed multimedia databases: Techniques and applications (pp 3757)... requirements to retrieve objects and few candidate objects are retrieved based on those requirements The user then can reuse suitable objects to refine the query and in that process reuse the underlying database resources that initially retrieved those images Video Information Retrieval For efficient video information retrieval, video data has to be manipulated properly Four retrieval techniques are: (1)... MPEG7 video content sets yield encouraging results, which prove the efficiency of our video indexing and abstraction scheme Chapter XII is titled An Overview of Video Information Retrieval Techniques Video information retrieval is currently a very important topic of research in the area of multimedia databases Plenty of research has been undertaken in the past decade to design efficient video information. .. of video data is being generated these days all over the world This requires efficient and effective mechanisms to store, access, and retrieve these data But the technology developed to date to handle those issues is far from the level of maturity required Video data, as we know, would contain image, audio, graphical and textual data The first problem is the efficient organization of raw video data. .. indices for the key frames and then shots; and (4) information retrieval, where a query in the form of input is provided by the user and then, based on this input, a search is carried out through the database to establish symmetry with the information in the database (Farag & Abdel-Wahab, 2003) Content-based image retrieval, which is essential for efficient video information retrieval, is emerging as... available features have to be extracted and then similarity would have to be established with the images of the video data for selection and retrieval The fourth problem is the effective and efficient data transmission through networking, which is addressed through Video- on-Demand (VoD) and Quality of Service (QoS) Also, there is the issue of data security, i.e., data should not be accessible to or downloadable . Deb University of Southern Queensland, Australia xii Acknowledgments Section I An Introduction to Video Data Management and Information Retrieval Video Data Management and Information Retrieval 1 Copyright. AND INFORMATION RETRIEVAL Chapter I Video Data Management and Information Retrieval 1 Sagarmay Deb, University of Southern Queensland, Australia S ECTION II: V IDEO DATA STORAGE TECHNIQUES AND. namely, video data management and video information retrieval. Both of these areas still need research efforts to seek solutions to many unresolved prob- lems for efficient data management and information