We investigate the stability of black hole solutions in an effective theory derived from a superstring model, which includes a dilaton field and the Gauss-Bonnet term. The critical solution, below which mass no static solution exists, divides a family of solutions in the mass-entropy diagram into two. The upper branch approaches the Schwarzschild solution in the large mass limit, while the lower branch ends up with a singular solution which has a naked singularity. In order to investigate the stability of black hole solutions, we adopt two methods. The first one is catastrophe theory, with which we discuss the stability of non-Abelian black holes in general relativity. The present system is classified as a fold catastrophe, which is the simplest case. Following catastrophe theory, if we regard entropy and mass as the potential and the control parameter, respectively, we find the lower branch is more unstable than the upper branch. To confirm this, we study the second method, which is a linear perturbation analysis. We find an unstable mode only for the solutions in the lower branch. Hence, our investigation presents one example that catastrophe theory is also applicable for a generalized theory of gravity.

• Received 19 February 1998

DOI:https://doi.org/10.1103/PhysRevD.58.084004