Normwise Condition of Linear Equation Solving

Abstract

Every invertible matrix $A\in \mathbb {R}^{n\times n}$ can be uniquely factored as A=QR, where Q is an orthogonal matrix and R is upper triangular with positive diagonal entries. This is called the QR factorization of A, and in numerical linear algebra, different ways for computing it are studied. From the QR factorization one obtains the solution of the system Ax=b by y=Q ^T b and x=R ⁻¹ y, where the latter is easily computed by back substitution.

The Householder QR factorization method is an algorithm for computing the QR-decomposition of a given matrix. It is one of the main engines in numerical linear algebra. The following result states a backward analysis for this algorithm.

Theorem 1.1  Let $A\in \mathbb {R}^{n\times n}$ be invertible and $b\in \mathbb {R}^{n}$. If the system Ax=b is solved using the Householder QR factorization method, then the computed solution $\tilde{x}$ satisfies

$$\tilde{A}\tilde{x}=\tilde{b}, $$

where $\tilde{A}$ and $\tilde{b}$ satisfy the relative error bounds

$$\|\tilde{A}-A\|_F\leq n\gamma_{cn}\|A\|_F \quad\mathit{and}\quad \|\tilde{b}-b\|\leq n\gamma_{cn}\|b\| $$

where $\gamma_{cn}:=\frac{cn\epsilon _{\mathsf {mach}}}{1- cn\epsilon _{\mathsf {mach}}}$ with a small constant c.  □

This yields $\|\tilde{A} - A \| \le n^{3/2} \gamma_{cn}\, \| A\|$ when the Frobenius norm is replaced by the spectral norm. It follows from this backward stability result that the relative error for the computed solution $\tilde{x}$ satisfies

$$ \frac{\|\tilde{x}-x\|}{\|x\|} \leq cn^{5/2}\epsilon _{\mathsf {mach}}\mathsf {cond}(A,b) +o(\epsilon _{\mathsf {mach}}), $$

(*)

and the loss of precision is bounded by

$$ \mathsf {LoP}\bigl(A^{-1}b\bigr) \leq \frac{5}{2}\log n+\log \mathsf {cond}(A,b) + \log c+o(1) . $$

Here cond(A,b) is the normwise condition number for linear equation solving,

$$\mathsf {cond}(A,b)=\lim_{\delta\to0} \sup_{\max\{\mathsf {RelError}(A),\mathsf {RelError}(b)\}\leq\delta} \frac{\mathsf {RelError}(A^{-1}b)}{\max\{\mathsf {RelError}(A),\mathsf {RelError}(b)\} } , $$

where RelError(A) is defined with respect to the spectral norm and RelError(b) with respect to the Euclidean norm. Inequality (*) calls for a deeper understanding of what cond(A,b) is than the equality above. The pursuit of this understanding is the goal of this chapter.