4 Chapter 1
neural network architectures, such as CNNs, RNNs, GNNs, and Transform-
ers. We will show that all of the above can be obtained by the choice of a
geometric domain, its symmetry group, and appropriately constructed invari-
ant and equivariant neural network layers—what we refer to as the ‘Geometric
Deep Learning blueprint.’ We will exemplify instances of this blueprint on the
‘5G of Geometric Deep Learning’: Graphs, Grids, Groups, Geometric Graphs,
and Gauges. On the other hand, Geometric Deep Learning gives a construc-
tive procedure to incorporate prior physical knowledge into neural networks
and provide a principled way to build new architectures. With this premise,
we shall now explore how the ideas central to Geometric Deep Learning have
crystallised through time.
1.1 On the Shoulders of Giants
“Symmetry, as wide or as narrow as you may define its meaning, is one idea
by which man through the ages has tried to comprehend and create order,
beauty, and perfection.” This somewhat poetic definition of symmetry is given
in the eponymous book of the great mathematician Hermann Weyl (1952),
his Schwanengesang on the eve of retirement from the Institute for Advanced
Study in Princeton. Weyl traces the special place symmetry has occupied in
science and art to the ancient times, from Sumerian symmetric designs to the
Pythagoreans who believed the circle to be perfect due to its rotational sym-
metry. Plato considered the five regular polyhedra bearing his name today so
fundamental that they must be the basic building blocks shaping the material
world.
Yet, though Plato is credited with coining the term σνµµϵτρ´ια, which lit-
erally translates as ‘same measure’, he used it only vaguely to convey the
beauty of proportion in art and harmony in music. It was the astronomer and
mathematician Johannes Kepler (1611) to attempt the first rigorous analysis of
the symmetric shape of water crystals. In his treatise ‘On the Six-Cornered
Snowflake’
1
he attributed the six-fold dihedral structure of snowflakes to
hexagonal packing of particles – an idea that though preceded the clear
understanding of how matter is formed, still holds today as the basis of
crystallography (Ball 2011).
In modern mathematics, symmetry is almost univocally expressed in the
language of group theory. The origins of this theory are attributed to Évariste
Galois, who coined the term and used it to study the solvability of polynomial
equations in the 1830s.
2
Two other names associated with group theory are
those of Sophus Lie and Felix Klein, who met and worked fruitfully together
for a period of time (Tobies 2019). The former would develop the theory of