Rotation-Invariant HOG Descriptors Using Fourier Analysis in Polar and Spherical Coordinates

Abstract

The histogram of oriented gradients (HOG) is widely used for image description and proves to be very effective. In many vision problems, rotation-invariant analysis is necessary or preferred. Popular solutions are mainly based on pose normalization or learning, neglecting some intrinsic properties of rotations. This paper presents a method to build rotation-invariant HOG descriptors using Fourier analysis in polar/spherical coordinates, which are closely related to the irreducible representation of the 2D/3D rotation groups. This is achieved by considering a gradient histogram as a continuous angular signal which can be well represented by the Fourier basis (2D) or spherical harmonics (3D). As rotation-invariance is established in an analytical way, we can avoid discretization artifacts and create a continuous mapping from the image to the feature space. In the experiments, we first show that our method outperforms the state-of-the-art in a public dataset for a car detection task in aerial images. We further use the Princeton Shape Benchmark and the SHREC 2009 Generic Shape Benchmark to demonstrate the high performance of our method for similarity measures of 3D shapes. Finally, we show an application on microscopic volumetric data.

Appendix

1.1 Computation of the Tensorial Harmonic Expansion

Given a spherical tensor field $\mathbf{F} \in \mathcal{T }^{\ell }$, we have a way to compute the tensorial harmonic expansion in Eq.(34), which is more efficient than the direct projections.

First we compute the scalar (SH) expansion on each individual tensor component ${F}_m:\mathbb{R }^3 \rightarrow \mathbb{C }$ as

$$\begin{aligned} {{{{F}}_m}(r,\theta ,\varphi )} = \sum _{j=0}^{\infty }\sum _{n=-j}^j{\overline{\hat{{b}}_{m,n}^{j}}(r) {Y}^j_n(\theta , \varphi ) }, \end{aligned}$$

(42)

then the tensorial expansion coefficients $\mathbf{a}^{j,k}(r)$ can be computed from the above component-wise expansions by a derived relation as

$$\begin{aligned} {a}_{m^{\prime }}^{j,k}(r)\!=\!\frac{2(j+k)+1}{2{\ell }\!+\!1} \sum _{m,n} {\hat{b}_{m,n}^{\,j}(r)} C(\ell ,m|j\!+\!k,m^{\prime },j,n),\nonumber \\ \end{aligned}$$

(43)

where $-(j+k) \le m^{\prime } \le j+k$. See Reisert and Burkhardt (2009) for proofs. We need to compute the ClebschGordan coefficients $C$ in this circumstance. An easy way is to use their relation to the Wigner 3-j symbols $\left( \begin{array}{lll} j_1&{}j_2&{}j_3\\ m_1&{}m_2&{}m_3 \end{array}\right)$ (Brink and Satchler 1968), which is written as

$$\begin{aligned}&C(j_3,m_3|j_1,m_1,j_2,m_2) = (-1)^{j_1-j_2+m_3} \sqrt{2j_3 + 1}\nonumber \\&\quad \times \left( \begin{array}{lll} j_1 &{} j_2 &{} j_3 \\ m_1 &{} m_2 &{} m_3 \end{array}\right) . \end{aligned}$$

(44)

One can use the function “gsl_sf_coupling_3j” in the GNU Scientific Library to compute the Wigner 3-j symbol.

1.2 Spherical Gaussian Derivatives

Let $\mathbf{F} \in \mathcal{T }_\ell$, the spherical up-derivative $\varvec{\nabla }^{1}_{}:\mathcal{T }_\ell \rightarrow \mathcal{T }_{\ell +1}$ and the down-derivative $\varvec{\nabla }^{}_{1}:\mathcal{T }_\ell \rightarrow \mathcal{T }_{\ell -1}$ (Reisert and Burkhardt 2009) are defined as

$$\begin{aligned} \varvec{\nabla }^{1}_{}\mathbf{F}&:= \varvec{\nabla }^{}_{} \bullet _{(\ell +1|1, \ell )} \mathbf{F}, \end{aligned}$$

(45)

$$\begin{aligned} \varvec{\nabla }^{}_{1}\mathbf{F}&:= \varvec{\nabla }^{}_{} \bullet _{(\ell -1|1, \ell )} \mathbf{F}, \end{aligned}$$

(46)

where $\nabla = (\frac{1}{\sqrt{2}}(\partial _x - \mathrm{i }{} \partial _y), \partial _z, -\frac{1}{\sqrt{2}}(\partial _x + \mathrm{i }{} \partial _y))$ is the spherical gradient operator with $\partial _x,\partial _y,\partial _z$ being the standard partial derivatives. It is further defined that $\varvec{\nabla }^{j_u}_{j_d}\mathbf{V} = \underbrace{\varvec{\nabla }^{}_{1}\ldots \varvec{\nabla }^{}_{1}}_{j_d \text{ times }}\underbrace{\varvec{\nabla }^{1}_{}\ldots \varvec{\nabla }^{1}_{}}_{j_u \text{ times }}\mathbf{V}$. One important property of this operation is that it maps a spherical tensor field to a higher or lower rank spherical tensor field. This is analogous to the fact that computing derivatives on a scalar field produces a gradient field, which is a rank-$1$ tensor, and a subsequent derivative can either produce the Hessian (rank-$2$ tensor) or the divergence (rank-$0$ tensor).

For $\mathbf{V} = \varvec{\nabla }^{1}_{} \mathbf{V}^{\prime } \nonumber$, where $\mathbf{V}^{\prime }:\mathbb{R }^3 \rightarrow {\mathbb{C }}^{2(\ell -1)+1}, \mathbf{V}:\mathbb{R }^3 \rightarrow {\mathbb{C }}^{2\ell +1}$, by indexing the elements of $\mathbf{V}$ and $\mathbf{V}^{\prime }$ as $\{V_{-\ell },\ldots ,V_{\ell }\}$ and $\{V^{\prime }_{-\ell +1},\ldots ,V^{\prime }_{\ell -1}\}$, the computation rule of $\varvec{\nabla }^{1}_{}$ is:

$$\begin{aligned} V_{m}&= w(\ell , m,-1) \; \frac{1}{\sqrt{2}} (\partial _x - \mathrm{i }{} \partial _y) V^{\prime }_{m+1} \nonumber \\&+ w(\ell , m,0)\;\partial _z V^{\prime }_m\nonumber \\&- w(\ell , m,1)\;\frac{1}{\sqrt{2}}(\partial _x + \mathrm{i }{} \partial _y) V^{\prime }_{m-1} \quad , \end{aligned}$$

(47)

where $w$ is the weighting coefficients which can be pre-computed from two Clebsch-Gordan coefficients as $w{(\ell , m, a)} = \frac{C(\ell ,m|\ell -1 ,m-a, 1,a)}{C(\ell ,0|\ell -1, 0, 1, 0)}$. Thus the computation of the spherical tensor derivatives is just a group of weighted combinations of normal Cartesian derivatives.

Equation (47) also fits the spherical down-derivative $\mathbf{V} = \varvec{\nabla }^{}_{1} \mathbf{V}^{\prime }$, where $\mathbf{V}:\mathbb{R }^3 \rightarrow {\mathbb{C }}^{2\ell +1}$ and $\mathbf{V}^{\prime }:\mathbb{R }^3 \rightarrow {\mathbb{C }}^{2(\ell +1)+1}$. The only difference are the coefficients: $w{(\ell , m, a)} = \frac{C(\ell ,m|\ell +1,m-a, 1,a)}{C(\ell ,0|\ell +1,0, 1,0)}$.

A fast filtering tool is derived by computing the derivatives on an isotropic Gaussian function, which creates a series of basis function of different tensor ranks, as $\varvec{\nabla }^{j_u}_{j_d} G \in \mathcal{T }_{j_u-j_d}$ (where $j_u \ge j_d, G$ is a Gaussian function). The convolution with the spherical Gaussian derivatives can be computed efficiently like the standard Gaussian derivatives based on the commutativity of the convolution and differentiation. As an example, let $\mathbf{F} \in \mathcal{T }_\ell$ be a spherical tensor field, we have

$$\begin{aligned} \varvec{\nabla }^{j_u}_{j_d} G \;\widetilde{\bullet }_{{(\ell +j_u-j_d| j_u-j_d, \ell )}} \;\mathbf{F} = \varvec{\nabla }^{j_u}_{j_d}(G \;\widetilde{\bullet }_{(\ell | 0, \ell )} \mathbf{F}). \end{aligned}$$

(48)

We can therefore compute multiple filtering outputs (for different $\{j_u,j_d\}$) by a single tensorial convolution plus differentiations. Note, the convolution like $G \;\widetilde{\bullet }_{(\ell | 0, \ell )} \;\mathbf{F}$ is equivalent to normal Gaussian convolutions as $[G \;\widetilde{\bullet }_{(\ell | 0, \ell )} \mathbf{F}]_m = G * F_m$ (because $C(\ell ,m|\ell ,m,0,0) = 1$). The output is a tensor field of rank $\ell + j_u - j_d$. In the context of this paper, we can take the SGD as derivatives after a scale-space selection by Gaussian convolution. The only important property for the rotation-invariance is that the introduced basis functions are spherical tensor fields.