Contents
Acknowledgement ix
Preface xi
Notation xvi
I GEOMETRIC FOUNDATIONS OF DEEP LEARNING 1
1 Introduction 3
1.1 On the Shoulders of Giants 4
1.2 Towards Geometric Deep Learning 10
1.3 The ‘Erlangen Programme’ of Deep Learning 29
2 Foundations of Supervised Learning 39
2.1 Main Ingredients of Supervised Learning 39
2.2 Decomposition of Risk 44
2.3 Over-parametrised Regime 47
2.4 The Curse of Dimensionality in High-Dimensional Learning 49
2.5 Linear Approximation with Positive-Definite Kernels 50
2.6 From Linear to Non-Linear Approximation 52
2.7 Approximation with Shallow Networks and Beyond 53
Exercises 56
3 Foundations of Equivariant Deep Learning 61
3.1 Symmetry Transformations 62
3.2 Symmetries of Learning Problems 64
3.3 Symmetry Groups 69
3.4 Group Actions and Representations 78
3.5 Equivariant Maps & Equivariant Nets 83
3.6 The Equivariance Cookbook 86
Exercises 97
4 Foundations of Geometric Deep Learning 101
4.1 Geometric Domains 102
vi Contents
4.2 Invariant Function Classes 106
4.3 Deformation Stability 110
4.4 Scale Separation 113
4.5 The GDL Blueprint 119
4.6 Summary 123
Exercises 124
II LEARNING ON GEOMETRIC DOMAINS 126
5 Graphs 129
5.1 Domain 130
5.2 Model 134
5.3 Case Study: Recommender Systems and Social Networks 151
5.4 Case Study: Large Language Models 154
Exercises 160
6 Grids 167
6.1 Domain 168
6.2 Model 172
6.3 Case Study: Long Short-Term Memory 182
Exercises 188
7 Group Convolution on Homogeneous Spaces 193
7.1 Domain 194
7.2 Model 198
7.3 Case Study: TacticAI 201
Exercises 206
8 Geometric Graphs 209
8.1 Domain 210
8.2 Model 214
8.3 Case Study: Manifold and Mesh CNNs 227
8.4 Case Study: AlphaFold 2 242
Exercises 251
III HORIZONS OF GEOMETRIC DEEP LEARNING 255
9 Further Beyond 257
9.1 Why go beyond groups? 258
9.2 What lies beyond? 262
9.3 Where does the group come from? 282
Contents vii
9.4 Closing remarks 285
Exercises 286
References 290