Shun-ichi Amari

Information Geometry and Its Applications

Andrzej Cichocki

Rafal Zdunek

Anh Huy Phan

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMF’s various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models. Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors’ own recently developed techniques in the subject area. Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms. Provides a comparative analysis of the different methods in order to identify approximation error and complexity. Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book. The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.

Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

Nonnegative Matrix and Tensor Factorizations

This is the first comprehensive book on information geometry, written by the founder of the field. It begins with an elementary introduction to dualistic geometry and proceeds to a wide range of applications, covering information science, engineering, and neuroscience. It consists of four parts, which on the whole can be read independently. A manifold with a divergence function is first introduced, leading directly to dualistic structure, the heart of information geometry. This part (Part I) can be apprehended without any knowledge of differential geometry. An intuitive explanation of modern differential geometry then follows in Part II, although the book is for the most part understandable without modern differential geometry. Information geometry of statistical inference, including time series analysis and semiparametric estimation (the Neyman–Scott problem), is demonstrated concisely in Part III. Applications addressed in Part IV include hot current topics in machine learning, signal processing, optimization, and neural networks. The book is interdisciplinary, connecting mathematics, information sciences, physics, and neurosciences, inviting readers to a new world of information and geometry. This book is highly recommended to graduate students and researchers who seek new mathematical methods and tools useful in their own fields.

Hiroshi Nagaoka

Information geometry provides the mathematical sciences with a new framework of analysis. It has emerged from the investigation of the natural differential geometric structure on manifolds of probability distributions, which consists of a Riemannian metric defined by the Fisher information and a one-parameter family of affine connections called the $\alpha$-connections. The duality between the $\alpha$-connection and the $(-\alpha)$-connection together with the metric play an essential role in this geometry. This kind of duality, having emerged from manifolds of probability distributions, is ubiquitous, appearing in a variety of problems which might have no explicit relation to probability theory. Through the duality, it is possible to analyze various fundamental problems in a unified perspective. The first half of this book is devoted to a comprehensive introduction to the mathematical foundation of information geometry, including preliminaries from differential geometry, the geometry of manifolds or probability distributions, and the general theory of dual affine connections. The second half of the text provides an overview of many areas of applications, such as statistics, linear systems, information theory, quantum mechanics, convex analysis, neural networks, and affine differential geometry. The book can serve as a suitable text for a topics course for advanced undergraduates and graduate students.

Methods of Information Geometry

From the reviews: "In this Lecture Note volume the author describes his differential-geometric approach to parametrical statistical problems summarizing the results he had published in a series of papers in the last five years. The author provides a geometric framework for a special class of test and estimation procedures for curved exponential families. ... ... The material and ideas presented in this volume are important and it is recommended to everybody interested in the connection between statistics and geometry ..." #Metrika#1 "More than hundred references are given showing the growing interest in differential geometry with respect to statistics. The book can only strongly be recommended to a geodesist since it offers many new insights into statistics on a familiar ground." #Manuscripta Geodaetica#2

Differential-Geometrical Methods in Statistics

With solid theoretical foundations and numerous potential applications, Blind Signal Processing (BSP) is one of the hottest emerging areas in Signal Processing. This volume unifies and extends the theories of adaptive blind signal and image processing and provides practical and efficient algorithms for blind source separation: Independent, Principal, Minor Component Analysis, and Multichannel Blind Deconvolution (MBD) and Equalization. Containing over 1400 references and mathematical expressions Adaptive Blind Signal and Image Processing delivers an unprecedented collection of useful techniques for adaptive blind signal/image separation, extraction, decomposition and filtering of multi-variable signals and data. Offers a broad coverage of blind signal processing techniques and algorithms both from a theoretical and practical point of view Presents more than 50 simple algorithms that can be easily modified to suit the reader's specific real world problems Provides a guide to fundamental mathematics of multi-input, multi-output and multi-sensory systems Includes illustrative worked examples, computer simulations, tables, detailed graphs and conceptual models within self contained chapters to assist self study Accompanying CD-ROM features an electronic, interactive version of the book with fully coloured figures and text. C and MATLAB user-friendly software packages are also provided MATLAB is a registered trademark of The MathWorks, Inc. By providing a detailed introduction to BSP, as well as presenting new results and recent developments, this informative and inspiring work will appeal to researchers, postgraduate students, engineers and scientists working in biomedical engineering, communications, electronics, computer science, optimisations, finance, geophysics and neural networks.

Shun-ichi Amari

Books

Your next favorite book is out there