A Kogbetliantz-type algorithm for the hyperbolic SVD

Abstract

In this paper, a two-sided, parallel Kogbetliantz-type algorithm for the hyperbolic singular value decomposition (HSVD) of real and complex square matrices is developed, with a single assumption that the input matrix, of order n, admits such a decomposition into the product of a unitary, a non-negative diagonal, and a J-unitary matrix, where J is a given diagonal matrix of positive and negative signs. When J = ±I, the proposed algorithm computes the ordinary SVD. The paper’s most important contribution—a derivation of formulas for the HSVD of 2 × 2 matrices—is presented first, followed by the details of their implementation in floating-point arithmetic. Next, the effects of the hyperbolic transformations on the columns of the iteration matrix are discussed. These effects then guide a redesign of the dynamic pivot ordering, being already a well-established pivot strategy for the ordinary Kogbetliantz algorithm, for the general, n × n HSVD. A heuristic but sound convergence criterion is then proposed, which contributes to high accuracy demonstrated in the numerical testing results. Such a J-Kogbetliantz algorithm as presented here is intrinsically slow, but is nevertheless usable for matrices of small orders.

Appendix: . Proofs of Lemmas 3.1 and 3.2

Proof

(Lemma 3.1) Let, for 1 ≤ ℓ ≤ n, $\gamma _{\ell }=\arg (x_{\ell })$ and $\delta _{\ell }=\arg (y_{\ell })$. Then,

$$ \left[\begin{array}{cc}x_{\ell}&y_{\ell} \end{array}\right]= \left[\begin{array}{cc}e^{\mathrm{i}\gamma_{\ell}}|x_{\ell}|&e^{\mathrm{i}\delta_{\ell}}|y_{\ell}| \end{array}\right] = e^{\mathrm{i}\gamma_{\ell}}\left[\begin{array}{cc}|x_{\ell}|&e^{\mathrm{i}(\delta_{\ell}-\gamma_{\ell})}|y_{\ell}| \end{array}\right]= e^{\mathrm{i}\delta_{\ell}}\left[\begin{array}{cc}e^{\mathrm{i}(\gamma_{\ell}-\delta_{\ell})}|x_{\ell}|&|y_{\ell}| \end{array}\right]. $$

Using the second equality, from the matrix multiplication it follows

$$ x_{\ell}^{\prime}=e^{\mathrm{i}\gamma_{\ell}^{}}(|x_{\ell}^{}|\cosh\psi+e^{\mathrm{i}(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)}|y_{\ell}^{}|\sinh\psi), $$

and using the third equality, from the matrix multiplication it follows

$$ y_{\ell}^{\prime}=e^{\mathrm{i}\delta_{\ell}^{}}(e^{\mathrm{i}(\gamma_{\ell}^{}-\delta_{\ell}^{}+\beta)}|x_{\ell}^{}|\sinh\psi+|y_{\ell}^{}|\cosh\psi). $$

Since $|e^{-\mathrm {i}\gamma _{\ell }^{}}x_{\ell }^{\prime }|=|x_{\ell }^{\prime }|$ and $|e^{-\mathrm {i}\delta _{\ell }^{}}y_{\ell }^{\prime }|=|y_{\ell }^{\prime }|$ and $\cos \limits (-\phi )=\cos \limits \phi$, it holds

$$ \begin{aligned} |x_{\ell}^{\prime}|^2 &=(|x_{\ell}^{}|\cosh\psi+\cos(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)|y_{\ell}^{}|\sinh\psi)^2+(\sin(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)|y_{\ell}^{}|\sinh\psi)^2\\ &=|x_{\ell}^{}|^2\cosh^2\psi+|y_{\ell}^{}|^2\sinh^2\psi+\cos(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)|x_{\ell}^{}||y_{\ell}^{}|2\cosh\psi\sinh\psi,\\ |y_{\ell}^{\prime}|^2 &=(\cos(\gamma_{\ell}^{}-\delta_{\ell}^{}+\beta)|x_{\ell}^{}|\sinh\psi+|y_{\ell}^{}|\cosh\psi)^2+(\sin(\gamma_{\ell}^{}-\delta_{\ell}^{}+\beta)|x_{\ell}^{}|\sinh\psi)^2\\ &=|x_{\ell}^{}|^2\sinh^2\psi+|y_{\ell}^{}|^2\cosh^2\psi+\cos(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)|x_{\ell}^{}||y_{\ell}^{}|2\cosh\psi\sinh\psi. \end{aligned} $$

After grouping the terms, the square of the Frobenius norm of the new ℓ th row is

$$ \begin{aligned} \left\|\left[\begin{array}{cc}x_{\ell}^{\prime} & y_{\ell}^{\prime} \end{array}\right]\right\|_{F}^{2}= |x_{\ell}^{\prime}|^{2}+|y_{\ell}^{\prime}|^{2}&= (\cosh^{2}\psi+\sinh^{2}\psi)(|x_{\ell}^{}|^{2}+|y_{\ell}^{}|^{2})\\ &\quad+2\cos(\delta_{\ell}^{}-\gamma_{\ell}^{}-\beta)|x_{\ell}^{}||y_{\ell}^{}|2\cosh\psi\sinh\psi. \end{aligned} $$

(A.1)

Summing the left side of (A.1) over all ℓ one obtains

$$ \left\|\left[\begin{array}{cc}\mathbf{x}^{\prime}&\mathbf{y}^{\prime} \end{array}\right]\right\|_F^2= \sum\limits_{\ell=1}^n\left( |x_{\ell}^{\prime}|^2+|y_{\ell}^{\prime}|^2\right), $$

what is equal to the right side of (A.1), summed over all ℓ,

$$ \sum\limits_{\ell=1}^n\left( (\cosh^2\psi+\sinh^2\psi)(|x_{\ell}^{}|^2+|y_{\ell}^{}|^2)+2\zeta_{\ell}^{}|x_{\ell}^{}||y_{\ell}^{}|2\cosh\psi\sinh\psi\right), $$

where $-1\le \zeta _{\ell }^{}=\cos \limits (\delta _{\ell }^{}-\gamma _{\ell }^{}-\beta )\le 1$, so $|\zeta _{\ell }^{}|\le 1$. The last sum can be split into a non-negative part and the remaining part of an arbitrary sign,

$$ (\cosh^2\psi+\sinh^2\psi)\sum\limits_{\ell=1}^n(|x_{\ell}^{}|^2+|y_{\ell}^{}|^2)+ 2\cosh\psi\sinh\psi\sum\limits_{\ell=1}^n 2\zeta_{\ell}^{}|x_{\ell}^{}||y_{\ell}^{}|. $$

Using the triangle inequality, and observing that ${\sum }_{\ell =1}^{n}(|x_{\ell }^{}|^{2}+|y_{\ell }^{}|^{2})=\left \|\left [\begin {array}{cc}\mathbf {x}&\mathbf {y} \end {array}\right ]\right \|_{F}^{2}$, this value can be bounded above by

$$ (\cosh^2\psi+\sinh^2\psi)\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2+ 2\cosh\psi|\sinh\psi|\sum\limits_{\ell=1}^n 2|x_{\ell}^{}||y_{\ell}^{}|, $$

and below by

$$ (\cosh^2\psi+\sinh^2\psi)\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2- 2\cosh\psi|\sinh\psi|{\sum}_{\ell=1}^n 2|x_{\ell}^{}||y_{\ell}^{}|, $$

where both bounds can be simplified by the identities

$$ \cosh^2\psi+\sinh^2\psi=\cosh(2\psi),\qquad 2\cosh\psi|\sinh\psi|=|\sinh(2\psi)|. $$

By the inequality of arithmetic and geometric means it holds $2|x_{\ell }^{}||y_{\ell }^{}|\le (|x_{\ell }^{}|^{2}+|y_{\ell }^{}|^{2})$, so a further upper bound is reached as

$$ \cosh(2\psi)\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2+ |\sinh(2\psi)|\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2, $$

and a further lower bound as

$$ \cosh(2\psi)\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2- |\sinh(2\psi)|\left\|\left[\begin{array}{cc}\mathbf{x}&\mathbf{y} \end{array}\right]\right\|_F^2, $$

what, after grouping the terms and dividing by $\left \|\left [\begin {array}{cc}\mathbf {x}&\mathbf {y} \end {array}\right ]\right \|_{F}^{2}$, concludes the proof. □

Proof

(Lemma 3.2) Note that $\cosh (2\psi )+|\sinh (2\psi )|\ge \cosh (2\psi )\ge 1$, and

$$ \begin{aligned} 1&=\cosh^2(2\psi)-\sinh^2(2\psi)\\ &=(\cosh(2\psi)-|\sinh(2\psi)|)\cdot(\cosh(2\psi)+|\sinh(2\psi)|)\\ &\ge\cosh(2\psi)-|\sinh(2\psi)|>0. \end{aligned} $$

If ψ = 0, the equalities in the bounds established in Lemma 3.1 hold trivially. Also, if both equalities hold simultaneously, ψ = 0.

The inequality of arithmetic and geometric means in the proof of Lemma 3.1 turns into equality if and only if |x_ℓ| = |y_ℓ| for all ℓ. When |x_ℓ||y_ℓ|≠ 0, it has to hold ζ_ℓ = ζ, where $\zeta =\pm \text {sign}(\sinh \psi )$, to reach the upper or the lower bound, respectively. From ζ = ± 1 it follows δ_ℓ = γ_ℓ + β + lπ for a fixed $l\in \mathbb {Z}$, i.e., $x_{\ell }=e^{\mathrm {i}\gamma _{\ell }}|x_{\ell }|$ and $y_{\ell }=\pm e^{\mathrm {i}\beta }e^{\mathrm {i}\gamma _{\ell }}|x_{\ell }|$, so y_ℓ = ±e^iβx_ℓ for all ℓ. Conversely, y = ±e^iβx implies, for all ℓ, that |x_ℓ| = |y_ℓ| and ζ_ℓ is a constant ζ = ± 1, so one of the two bounds is reached. □