Machine Learning Applications of Fixed Point Theory
Danilo Mandic (Imperial College London)
Abstract: Fixed point theory (FPT) originates from the Babylonian times, and has since been, implicitely or explicitely, present in some ordinary calculations in everyday life. Rigorous mathematical foundations of FPT have been established in the 1920s in the work of Stefan Banach and his collaborators. The associated contraction mapping theorems (CMT) allow for the formal analysis of the existence and uniqueness of fixed points and rates of convergence of fixed point iterations (FPI). In this tutorial, we will introduce the basics of FPT and illustrate the usefulness of employing this framework in some modern machine learning applications. FPI is based on the iterative approach and can be used on its own or conveniently combined with the standard recursive online learning. In addition, the treatment of nonlinearities in this framework is very natural, and precise stability bounds on system parameters can be derived and monitored. The tutorial is supported by case studies which include examples from the areas of i) analysis of stability of nonlinear systems, ii) convergence of adaptive filters, iii)analysis of data reusing algorithms, iv) image processing, v) other applications. Simulations based on practical problems in acoustic echo cancellation, computer vision, autonomous systems, and classification support this tutorial.