How We Implemented Our Three Families of LieBN to SPD Neural Networks

Table of Links

Abstract and 1 Introduction

2 Preliminaries

3. Revisiting Normalization

3.1 Revisiting Euclidean Normalization

3.2 Revisiting Existing RBN

4 Riemannian Normalization on Lie Groups

5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds

5.2 LieBN on SPD Manifolds

6 Experiments

6.1 Experimental Results

7 Conclusions, Acknowledgments, and References

APPENDIX CONTENTS

A Notations

B Basic layes in SPDnet and TSMNet

C Statistical Results of Scaling in the LieBN

D LieBN as a Natural Generalization of Euclidean BN

E Domain-specific Momentum LieBN for EEG Classification

F Backpropagation of Matrix Functions

G Additional Details and Experiments of LieBN on SPD manifolds

H Preliminary Experiments on Rotation Matrices

I Proofs of the Lemmas and Theories in the Main Paper

6 EXPERIMENTS

In this section, we implement our three families of LieBN to SPD neural networks. Following the previous work (Huang & Van Gool, 2017; Brooks et al., 2019b; Kobler et al., 2022a), we adopt three different applications: radar recognition on the Radar dataset (Brooks et al., 2019b), human action recognition on the HDM05 (M ¨uller et al., 2007) and FPHA (Garcia-Hernando et al., 2018) datasets, and EEG classification on the Hinss2021 dataset (Hinss et al., 2021). More details on datasets and hyper-parameters are exposed in App. G. Besides SPD neural networks, we also implement LieBN on special orthogonal groups and present some preliminary experiments (see App. H).

Implementation details: Note that our LieBN layers are architecture-agnostic and can be applied to any existing SPD neural network. In this paper, we focus on two network architectures: SPDNet