Seminar: "The dynamics of AI"

AUEB STATISTICS SEMINAR SERIES APRIL 2023

Panayotis Mertikopoulos, Department of Mathematics, National and Kapodistrian University of Athens

Title: The dynamics of AI

ROOM Τ102, NEW AUEB BUILDING

ABSTRACT:

The recent surge of breakthroughs in machine learning and artificial intelligence has brought to the forefront a tremendous need for new mathematics to serve both as a solid theoretical foundation and as a springboard for further developments. In this talk, we will focus on how machine learning models are actually trained to make predictions and/or generate new data, a problem which is intimately related to the mathematical theory of dynamical systems – and, in particular, the study of gradient flows and (stochastic) gradient descent. We will begin by discussing how dynamical systems (in both discrete and continuous time) can be used to analyze and predict the outcome of the training process of an artificial neural network, guaranteeing convergence to critical points while avoiding unstable saddle points and other undesirable solutions. We will then proceed to examine what type of phenomena may arise when such systems interact – e.g., as in the case of generative adversarial networks. In this more general setting, the convergence landscape is considerably more treacherous, and gradient algorithms may be trapped by "spurious attractors" that are in no way optimal - a fact which highlights the fundamental gap in difficulty between training generative versus discriminative models.

The pdf of the presentation can be found here.