Pattern Recognition, Third Edition

Pattern Recognition, Third Edition
| 2006-02-24 00:00:00 | | 0 | Artificial Intelligence


A classic -- offering comprehensive and unified coverage with a balance between theory and practice!

Pattern recognition is integral to a wide spectrum of scientific disciplines and technologies including image analysis, speech recognition, audio classification, communications, computer-aided diagnosis, and data mining. The authors, leading experts in the field of pattern recognition, have once again provided an up-to-date, self-contained volume encapsulating this wide spectrum of information.

Each chapter is designed to begin with basics of theory progressing to advanced topics and then discusses cutting-edge techniques. Problems and exercises are present at the end of each chapter with a solutions manual provided via a companion website where a number of demonstrations are also available to aid the reader in gaining practical experience with the theories and associated algorithms.

This edition includes discussion of Bayesian classification, Bayesian networks, linear and nonlinear classifier design (including neural networks and support vector machines), dynamic programming and hidden Markov models for sequential data, feature generation (including wavelets, principal component analysis, independent component analysis and fractals), feature selection techniques, basic concepts from learning theory, and clustering concepts and algorithms. This book considers classical and current theory and practice, of both supervised and unsupervised pattern recognition, to build a complete background for professionals and students of engineering.

FOR INSTRUCTORS: To obtain access to the solutions manual for this title simply register on our textbook website (textbooks.elsevier.com)and request access to the Computer Science or Electronics and Electrical Engineering subject area. Once approved (usually within one business day) you will be able to access all of the instructor-only materials through the `Instructor Manual` link on this book's full web page.

* The latest results on support vector machines including v-SVM's and their geometric interpretation
* Classifier combinations including the Boosting approach
* State-of-the-art material for clustering algorithms tailored for large data sets and/or high dimensional data, as required by applications such as web-mining and bioinformatics
* Coverage of diverse applications such as image analysis, optical character recognition, channel equalization, speech recognition and audio classification

User review
A very comprehensive book about pattern recognition techniques
Pattern Recognition by Theodoridis and Koutroumbas is ideal for anyone who wishes to have a wide overview of pattern recognition and machine learning schemes. The book is organized very well and provides a very good stand-alone insight into the corresponding subjects.

User review
not intuitive enough
Just a quick browse through, I find that the materials are not intuitive

enough. I tried to look for the explanation for Figure 6.21, but did not

find clear explanation. Some of the deeper stuff probably can be

generated by readers once the basic stuff is discussed in detail and

intuitively. In general, for someone with an excellent math background

tries to go into the pattern recongnition field, this is NOT the book.



User review
Excellent
Many who work in artificial intelligence have commented that it is the ability of the human brain to engage in pattern recognition that gives it true intelligence. Without a quantitative measure of machine intelligence it is difficult to assess this claim, but there is no doubt that being able to implement pattern recognition and classification in a machine in a manner that enables it to distinguish objects, find profitable patterns in financial time series, teach itself how to play a game by examining the moves, identify subsequences in genome data, identify malicious behavior in networks, and detect fraudulent behavior in mortgage contracts would be a major advance in artificial intelligence and also a profitable one from a financial standpoint. Even if the machine required assistance from a human to do these tasks it would still be very useful. If it were able to do them on its own without any supervision one could justifiably describe it as being more intelligent than one that required such supervision (the counterexample to this imputation of intelligence is simple trial-and-error, which of course is unsupervised).


This book is a formal treatment of pattern recognition that is geared to a readership with a strong mathematical background and which makes as its major theme the difference between `supervised' and `unsupervised' pattern recognition, with this difference sometimes being more qualitative than what one would like. In the introduction to the book the authors make clear the distinction between these approaches, motivate the problem of the classification of features, and outline briefly the stages in the design of a pattern classification system. As is well known, supervised pattern recognition involves the use of training data, whereas unsupervised pattern recognition does not. In the latter case, it is left to the machine to find similarities in the feature vectors, and then cluster the similar feature vectors together. Researchers in the field of pattern recognition have devised an enormous number of algorithms and reasoning patterns to perform both unsupervised and supervised learning, and they have not necessarily developed these approaches in the context of machine intelligence. Thus the book could also be viewed as a mathematical theory of pattern recognition instead of one that is embedded in the field of artificial intelligence. However it is classified it is a useful and important work, and is well worth the time taken to read and study.


One of the most interesting (and esoteric) discussions is found in chapter 15 of the book. One of these concerns algorithms for `competitive learning' wherein representatives are designated and then `compete` with each other after a feature vector X is presented to the algorithm. The `winner` is the representative that is closer to X and the representatives are then updated by moving the winner toward X, with the rest remaining constant or move toward X at a slower rate. The competitive learning algorithm is parametrized by the learning rates of the winner and the losers, and the losers can have different learning rates. The investigator however selects the values of these parameters beforehand, and therefore competitive learning strictly speaking should not be classified as totally unsupervised. To be really unsupervised the competitive learning algorithm would have to make the selection of these parameters and tune them as needed to reach the convergence criterion. The authors do discuss briefly a version of the algorithm where the learning rate is variable, but the rate is still subject to certain constraints. Chapter 15 also contains a brief discussion of the use of genetic algorithms in clustering.


Another topic in the book that is both interesting and important and is still surprisingly unknown by many is that of `independent component analysis'. Independent component analysis (ICA) is a generalization of principal component analysis in that it tries to find a transformation that takes a feature vector into one whose components are mutually independent, instead of merely decorrelated. All of the random variables must be non-Gaussian in order for this technique to work, since the Gaussian case gives back the usual principal component analysis. Independent component analysis is beginning to be applied to many different areas, including finance, risk management, medical imaging, and physics. It remains to see whether it will become a standardized tool in the many mathematical and statistical software packages that exist at the present time. The authors discuss two different ways to perform independent component analysis, one being an approach based on higher order cumulants, and the other, interestingly, on mutual information. In the latter approach, the mutual information between the transformed components is calculated to be the Kullback-Leibler probability distance between the joint probability distribution of the transformed components and the product of the marginal probability densities. This distance is of course zero if the components are statistically independent. The strategy is then to find the transformational matrix that minimizes the mutual information, since this will make the components maximally independent. As the authors point out, the problem with this approach is that the elements of the transformation matrix are hidden in the marginal probability distribution functions of the transformed variables. They then outline an approach that allows them to calculate the mutual information with the assumption that the transformation matrix is unitary.



User review
centred around clustering methods
[A review of the 3rd EDITION 2006.]


The authors give us an indepth survey of pattern recognition methods. All sorts of ideas. Like using a neural network approach with a multilayer perceptron that has back propagation implemented. Or using a Bayesian to classify and infer. Nor do they neglect support vector machines, which is a relatively recent idea that has gained some adherants.


Much of the text centres on clustering algorithms. Sequential and hierarchical, amongst others. Notice that many of the clusters found are rather subjective. Often depending on some initial choice of parameters. Here is one place where you might have to use your expert knowledge, in choosing some clustering method that yields reasonable results for your application.

User review
Pattern Recognition
Professor Theodoridis has written an exciting new book on pattern recognition. The topic is sometimes neglected, particularly in the fields of biomedical and electrical engineering, but it is essential to the understanding of signal and image shape on a mathematical basis, including similarities and differences in shape as well as how to extract, recognize, and measure the important components. Professor Theodoridis covers all of the classic steps in pattern recognition in great detail and in a readily understood fashion: sensors and pattern extraction, features extraction and selection, clustering, classification, supervised and unsupervised recognition, and evaluation of the system. Each section is backed up with computer simulation examples so that the reader can gain practical experience while reading the book. The author discusses essential concepts for computer programming of the pattern recognition techniques that are discussed. This work is necessarily mathematical, and therefore will tend to be of greatest interest to advanced students and practicing engineers in a variety of fields. Biomedical engineering is a rapidly expanding field that is key to the improvement of health care quality. There are plenty of biomedical examples including those in the section of the book on computer-aided diagnosis, such as for the detection of cancerous lesions in x-ray mammography. The section on speech recognition will be useful to engineers who are designing turnkey pattern recognition systems that include speech recognition as input and/or for use as a security key. Also included in the work are the most recently developed topics of interest including fuzzy clustering algorithms, and neural networks using genetic and annealing methods. This comprehensive work should prove to be an invaluable tool for the library of design engineers who work with signals and images. I heartily recommend it to all with a basic engineering background.


Edward Ciaccio, PhD

Assoc. Professor of Biomedical Engineering

Columbia University in New York