BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//wp-events-plugin.com//6.4.7.2//EN
TZID:Europe/Paris
X-WR-TIMEZONE:Europe/Paris
BEGIN:VEVENT
UID:0-930@lptms.universite-paris-saclay.fr
DTSTART;TZID=Europe/Paris:20240206T110000
DTEND;TZID=Europe/Paris:20240206T120000
DTSTAMP:20240131T095410Z
URL:http://www.lptms.universite-paris-saclay.fr/seminars/seminaire-du-lptm
s-giovanni-catania-ucm/
SUMMARY:Séminaire du LPTMS : Giovanni Catania (UCM) - Salle des séminaire
s du FAST et du LPTMS\, bâtiment Pascal n°530 - 6 Fév 24 11:00
DESCRIPTION:The Copycat Perceptron: Smashing Barriers Through Collective Le
arningGiovanni Catania (Complutense University of Madrid)The context of th
is talk is related to inference problems in the so-called teacher-student
scenario (or planted setting)\, where the goal is to retrieve a ground-tru
th signal (specified by a teacher) from a set of - possibly noisy\, or inc
omplete - data provided to a student. This setting offers a natural connec
tion with Bayesian inference and statistical physics\, which allows to det
ermine specific algorithmic thresholds separating regimes - or phases in a
thermodynamic sense - where signal recovery is possible or not. Among the
class of inference problems that are typically studied\, some of them are
characterized by a "hard" inference regime\, a mechanism related to the p
resence of first-order transitions where any polynomial optimization algor
ithm fails to find the solution. I this talk\, I will focus on a famous ex
ample of an inference problem with an inference-hard phase\, namely the bi
nary perceptron in the teacher-student scenario\, and the algorithmic prop
erties of a specific optimization algorithm - Simulated Annealing (SA) - i
n its ability to find the planted configuration (i.e\, the teacher's weigh
t vector)\, depending on the fraction of examples provided to the student
and an external thermal noise. It is well known how SA is sensible to the
presence of metastable states that can trap the dynamics into local minima
with poor generalization performances\; in this model\, such a phenomenon
occurs even in a portion of the thermodynamically easy-inference phase\,
where other Bayes-optimal algorithms are supposed to find the solution in
polynomial time . In a different inference problem\, it has recently been
conjectured that replicating the systems and adding a ferromagnetic intera
ction between real replicas favours the resulting algorithm (Replicated SA
\, or RSA) to find solutions where the single system does not\, as a conse
quence of a peculiar modification of the free energy landscape that allows
the coupled system to avoid these metastable local minima. I will discuss
how to characterize the phase diagram of a system made of several coupled
perceptron students using techniques stemming from the statistical physic
s of spin-glasses\, and how it modifies when considering a thermal noise a
ffecting the generation capabilities of each of them. In this context\, th
e replicated system can be considered as a model of collective learning am
ong a class of students that try to learn the teacher's rule while "cooper
ating" with each other. These results provide additional analytic and nume
rical evidence for the recently conjectured Bayes-optimal property of Repl
icated Simulated Annealing (RSA) for a sufficient number of replicas. From
a learning perspective\, these results also suggest that multiple student
s working together (in this case reviewing the same data) are able to lear
n the same rule both faster and with fewer examples\, a property that coul
d be exploited in the context of federated learning. Reference: G. Catania
\, A. Decelle\, B. Seoane\, arXiv:2308.03743
CATEGORIES:seminars
LOCATION:Salle des séminaires du FAST et du LPTMS\, bâtiment Pascal n°53
0\, rue André Riviere\, Orsay\, 91405\, France
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=rue André Riviere\, Orsay\
, 91405\, France;X-APPLE-RADIUS=100;X-TITLE=Salle des séminaires du FAST
et du LPTMS\, bâtiment Pascal n°530:geo:0,0
END:VEVENT
BEGIN:VTIMEZONE
TZID:Europe/Paris
X-LIC-LOCATION:Europe/Paris
BEGIN:STANDARD
DTSTART:20231029T020000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
END:STANDARD
END:VTIMEZONE
END:VCALENDAR