Statistical Physics Meets Machine Learning I

Presentations

• Towards a theory of deep learning for hierarchical and compositional data

  ORAL · Invited

  March 18, 2025, 8:00 AM – March 18, 2025, 8:36 AM

  Publication: Towards a theory of how the structure of language is acquired by deep neural networks (https://arxiv.org/abs/2406.00048)

  Presenters

  • Francesco Cagnetta

    • EPFL
    • Scuola Internazionale Superiore di Studi Avanzati (SISSA)

  Authors

  • Francesco Cagnetta

    • EPFL
    • Scuola Internazionale Superiore di Studi Avanzati (SISSA)

  View abstract →

• Locating Information in Large Language Models via Random Matrix Theory

  ORAL

  March 18, 2025, 8:36 AM – March 18, 2025, 8:48 AM

  Publication: https://arxiv.org/html/2410.17770v1

  Presenters

  • Bernd Rosenow

    • University Leipzig
    • University of Leipzig
    • Leipzig University

  Authors

  • Bernd Rosenow

    • University Leipzig
    • University of Leipzig
    • Leipzig University

  • Max Staats

    • University of Leipzig

  • Matthias Thamm

    • Leipzig University

  View abstract →

• Explaining High-order Interactions in Protein Language Models

  ORAL

  March 18, 2025, 8:48 AM – March 18, 2025, 9:00 AM

  Publication: Tsui, Darin, and Amirali Aghazadeh. "On Recovering Higher-order Interactions from Protein Language Models." arXiv preprint arXiv:2405.06645 (2024).

  Presenters

  • Amirali Aghazadeh

    • Georgia Institute of Technology

  Authors

  • Amirali Aghazadeh

    • Georgia Institute of Technology

  • Darin Tsui

    • Georgia Tech

  View abstract →

• Same features, different encodings: three case studies of path dependence in grokking and learning.

  ORAL

  March 18, 2025, 9:00 AM – March 18, 2025, 9:12 AM

  Presenters

  • Dmitry Manning-Coe

    • University of Illinois at Urbana-Champaign

  Authors

  • Dmitry Manning-Coe

    • University of Illinois at Urbana-Champaign

  • Jacopo Gliozzi

    • University of Illinois at Urbana-Champaign

  • Alexander G Stapleton

    • Queen Mary University of London

  • Edward Hirst

    • Queen Mary University of London

  • Marc Klinger

    • University of Illinois at Urbana-Champaign

  • Guiseppe de Tomasi

    • University of Illinois Urbana-Champaign

  • David S Berman

    • Queen Mary University of London

  View abstract →

• The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold

  ORAL · Invited

  March 18, 2025, 9:12 AM – March 18, 2025, 9:48 AM

  Publication: 1. Mao, J., Griniasty, I., Teoh, H.K., Ramesh, R., Yang, R., Transtrum, M.K., Sethna, J.P. and Chaudhari, P., 2024. The training process of many deep networks explores the same low-dimensional manifold. Proceedings of the National Academy of Sciences, 121(12), p.e2310002121.
  2. Ramesh, R., Mao, J., Griniasty, I., Yang, R., Teoh, H.K., Transtrum, M., Sethna, J.P. and Chaudhari, P., 2023. A picture of the space of typical learnable tasks. Proc. of International Conference of Machine Learning (ICML).

  Presenters

  • Itay Griniasty

    • Cornell University

  Authors

  • Itay Griniasty

    • Cornell University

  View abstract →

• Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron

  ORAL

  March 18, 2025, 9:48 AM – March 18, 2025, 10:00 AM

  Publication: https://arxiv.org/abs/2409.03749

  Presenters

  • Christian Schmid

    • University of Oregon

  Authors

  • Christian Schmid

    • University of Oregon

  • James M Murray

    • University of Oregon

  View abstract →

• Entropy Advantage in Neural Networks Generalizability

  ORAL

  March 18, 2025, 10:00 AM – March 18, 2025, 10:12 AM

  Presenters

  • Entao Yang

    • Air Liquide USA
    • Air Liquide

  Authors

  • Entao Yang

    • Air Liquide USA
    • Air Liquide

  • Xiaotian Zhang

    • City University of Hong Kong

  • Yue Shang

    • University of Pennsylvania

  • Ge Zhang

    • City University of Hong Kong

  View abstract →

• Temperature-tuning trained energy functions improves generative performance

  ORAL

  March 18, 2025, 10:12 AM – March 18, 2025, 10:24 AM

  Presenters

  • Peter Fields

    • University of Chicago

  Authors

  • Peter Fields

    • University of Chicago

  • Vudtiwat Ngampruetikorn

    • University of Sydney

  • David J Schwab

    • CUNY Graduate Center
    • The Graduate Center, CUNY
    • CUNY

  • Stephanie E Palmer

    • University of Chicago

  View abstract →

• Learning continuous spin models with real-valued restricted Boltzmann machines

  ORAL

  March 18, 2025, 10:24 AM – March 18, 2025, 10:36 AM

  Publication: https://arxiv.org/abs/2409.20377

  Presenters

  • Kai Zhang

    • University of Texas at Tyler

  Authors

  • Kai Zhang

    • University of Texas at Tyler

  View abstract →

• Abstract Withdrawn

  ORAL · Withdrawn

  March 18, 2025, 10:36 AM – March 18, 2025, 10:48 AM

  View abstract →