?
Equivariant Neural Networks with Geometric Algebras: A New Approach
This work is devoted to construction and implementation of new equivariant neural networks based on geometric (Clifford) algebras. We propose, implement, test, and compare with competitors a new architecture of equivariant neural networks, which we call Generalized Lipschitz Group Equivariant Neural Networks (GLGENN). These networks are equivariant to all pseudo-orthogonal transformations, including rotations. We introduce generalized Lipschitz groups and prove for the first time that the following mappings in geometric algebras are equivariant with respect to these groups and pseudo-orthogonal and complex orthogonal groups: projections onto subspaces determined by grade involution and reversion and polynomials of geometric algebra elements. Leveraging these equivariant mappings, we design generalized geometric product and linear layers. GLGENN demonstrate superior performance in benchmark equivariant regression tasks, outperforming competitors, while using fewer optimizable parameters. Due to a relatively small number of parameters in the architecture, GLGENN have less tendency to overfitting. GLGENN have promising applications in natural science and computer vision, where tasks inherently involve equivariance to pseudo-orthogonal transformations. Code is available at https://github.com/katyafilimoshina/glgenn-core.