1 Introduction

Now, there is a large number of observational data indicating the existence of dark matter (DM), such as the cosmic microwave background radiation (CMBR), the rotation curves (RC) of the galaxies and the large-scale structure of the universe. Based on these observational data, astronomers have proposed lots of dark matter models. Among all these dark matter models, cold dark matter (CDM) model [1, 2] and scalar field dark matter (SFDM) model have received much attention. In these dark matter models, the most important part is to obtain the spatial density distribution function of dark matter. At present, the distribution of dark matter in the large-scale structure of galaxies is clear [3]. However, the distribution of dark matter around supermassive black holes (BHs) is unclear [4]. Therefore, studying the distribution of dark matter around black holes is a very significant project. The dark matter around the black hole is usually characterized by its density parameter and characteristic radius. It is generally believed that the distribution of dark matter around black holes is a big “spike” [5,6,7]. Fortunately, with our efforts, we have derived black hole (BH) spacetime metrics both in a dark matter halo [8] and a dark matter spike [9], respectively, and generalized them to the case of rotation. In addition, through the black hole photos released by the Event Horizon Telescope (EHT) [10] and the observation of gravitational waves by the Laser Interferometer Gravitational Wave Observatory Scientific Collaboration and Virgo Collaboration (LIGO Scientific/Virgo) [11,12,13,14], the existence of black holes can be basically determined.

On the other hand, the term “black hole” was originally coined by Wheeler [15]. For a black hole, he proposed the famous “No-hair theorem” [16]. The object described by the No-hair theorem is a static isolated black hole. However, in our universe, it is hard to find an isolated black hole, suggesting that there may be various complex matter fields around the black hole, causing the black hole to be in a perturbed state. When a black hole is perturbed by the matter field, its initial perturbation can be represented by a complex frequency of excited oscillation mode, which is so-called quasinormal modes (QNM) [17]. This perturbation is usually divided into three stages, the second stage is the QNM. Vishveshwara show that when a black hole is in a perturbed state, it can emit gravitational waves [18]. And this kind of gravitational wave is just dominated by the QNM with a complex frequency. The real part of this frequency represents the oscillation frequency of the black hole when perturbed, while the imaginary part represents the rate of oscillation, also known as damping [19]. This QNM mode is directly related to the oscillating properties of black holes, so the study of QNM is helpful to understand the information about black holes. Besides, this QNM is an important source of gravitational waves from the supernova collapse. Kostas D. Kokkotas et al. show that the gravitational collapse of massive rotating stars and the merger of the binary star system can generate gravitational waves which are directly related to the QNM [20]. Now, there are many methods to calculate the QNM, such as WKB method [21, 22], Pöschl–Teller potential approximation [23,24,25] and continued fraction method [26, 27]. The QNM is a characteristic “sound” of a black hole which can provide us with a new method to verify a black hole in our universe [19].

Besides, according to the fact that no matter can escape from a black hole, the solution at the event horizon of the black hole is a pure incoming wave. However, for the behavior of the solution of the equation at infinity, it can be divided into two types. One of types is QNM we introduced before and another is the quasibound states (QBS). The solution of QNM is a pure outgoing wave when it is far away from the black hole, while QBS is only shown as an exponential decay when it is far away from the black hole [27]. According to the superradiant phenomenon, a rotating black hole produces wave amplification, and when the perturbation field is large, the superradiant phenomenon can cause instability [28]. This superradiant instability is closely related to the existence of QBS, which may have important astrophysical implications. Besides, if a rotating black hole is trapped inside an perfectly reflective cavity, it can cause a phenomenon called the “black hole bomb” [29, 30]. That’s because any initial perturbations would be magnified near the black hole and reflected back to the mirror, creating instability, which grows exponentially with the time. Through this amplified scattered wave, we can extract certain rotational energy from the ergosphere of the black hole. However, it should be noted that the rotation energy of a black hole is limited, and the process of extracting energy should take into account the backreaction effects. In fact, these studies show that Kerr black holes are prone to instabilities under massive fields because the mass term effectively suppresses the field [31,32,33,34]. Here, what we are interested in is whether the massive field of the Kerr-like black hole in the dark matter halo will also produce the instability similar to the black hole bomb. This is one of our study points.

In recent years, the QBS of black holes [35,36,37,38,39,40,41] and the QNM [42,43,44,45,46,47,48,49,50,51,52,53] have been extensively studied. Among the recent researches connected to our work, we list the following papers. In Ref. [27], they study the instabilities of rotating black holes in the massive scalar field. Richartz et al. use the scalar field perturbation to study the eigenfrequencies of the Kerr-like black holes [28]. Besides, with the release of black hole photos, people are also increasingly concerned about the interaction of other celestial bodies (or matter) around the black hole [54, 55]. Cardoso et al. introduce an exact solution for a black hole immersed in a galactic-like distribution of matter and use gravitational perturbations to study the quasinormal modes of this black hole [56]. Based on this, Konoplya studies the matter field perturbations, the greybody factors and the Unruh temperature [57] of the exact solution of this black hole. On the other hand, an interesting and important question is how these parameters of dark matter affect the QNM/QBS. These dark matter parameters generally come from different galaxies, and they are usually the results obtained through fitting processing on the basis of observation data [58, 59]. In recent years, the research on black hole immersed in dark matter and Schwarzschild black has received extensive attention [60,61,62,63,64]. Zhang et al. studied the QNM of a spherically symmetric black hole in a dark matter halo using gravitational perturbation [65, 66]. However, there are relatively few studies between black hole in dark matter and Kerr black hole. Comparing the differences between them is also a meaningful and important work. As the QNM/QBS of the Kerr black hole have become more familiar, these comparisons can help us understand black holes in a dark matter halo to some extent. So, in this paper, we will also try to study these questions. In the longer term, with the rapid development of gravitational wave detectors in the future, it is possible to detect signals from such black holes immersed in dark matter, which also provides an effective method for testing dark matter models.

In this paper, we mainly use the scalar field perturbation to study QNM/QBS of the Kerr-like black holes in a dark matter halo, and use the continued fraction method to calculate QNM/QBS frequencies. For the same time, these results are compared with the Kerr black hole. We will also analyze and verify the existence of the superradiant instabilities of black holes. In addition, we will also study the impacts of dark matter parameters on the QNM/QBS of black holes at the specific circumstances. This work is an in-depth study based on our previous work [67,68,69]. Although these previous works are on spherically symmetric black holes, these valuable experiences give us enough confidence to solve the case of axisymmetric black holes.

This paper is organized as follows. In Sect. 2, we briefly introduce the spacetime properties of Kerr-like black holes in a dark matter halo, and the discussion of extremal black holes of them. In particular, we unify the dark matter parameters into the black hole units. In Sect. 3, we introduce scalar field perturbation of this Kerr-like black holes, and show that how to use ansatz to decompose the complex perturbation equations into radial and angular equations. In Sect. 4, we mainly introduce the continued fraction method, which includes the asymptotic solution of the Kerr-like black hole oscillation behavior at the specific boundary conditions, the derivation of the 3-term recurrence formula of the radial/angular equations and the continued fraction equation. In Sect. 5, we mainly use the continued fraction method to calculate the QNM/QBS frequencies of the Kerr-like black holes both in CDM/SFDM models, and compare their results with Kerr black hole. At the same time, we will verify the existence of superradiant instabilities and give the figures of the maximum instabilities of the Kerr-like black holes in a dark matter halo. In addition, we will also study the impacts of dark matter parameters on the QNM/QBS of black holes at the specific circumstances. Finally, Sect. 6 is our conclusions and discussions. In this paper, we use mostly the black hole units that \(G=c=M_\text {BH}=r_{\text {BH}}=1\) and the radius of black hole (BH) is given by \(r_{\text {BH}}=GM_\text {BH}/c^2\).

2 The spacetime of the Kerr-like BHs in a dark matter halo

In this section, we will review the Kerr-like black hole metrics we obtained in a dark matter halo [8]. Both of them have the following form in four-dimensional coordinates,

$$\begin{aligned} \begin{aligned} ds^{2}=&-\left( 1-\frac{r^{2}+2Mr-r^{2}f(r)}{\Sigma ^{2}}\right) d t^{2} \\&+\frac{\Sigma ^{2}}{\Delta } d r^{2}+\Sigma ^{2} d \theta ^{2}+\frac{A\sin ^{2} \theta }{\Sigma ^{2}} d \phi ^{2} \\&-\frac{2 (r^{2}+2Mr-r^{2}f(r)) a \sin ^{2} \theta }{\Sigma ^{2}} d \phi d t, \end{aligned} \end{aligned}$$
(2.1)

with

$$\begin{aligned} \Delta= & {} r^{2}f(r)-2Mr+a^{2}, \quad \Sigma ^2=r^2+a^2\cos (\theta )^2,\nonumber \\ A= & {} \left( r^{2}+a^{2}\right) ^{2}- a^{2} \Delta \sin ^{2} \theta , \end{aligned}$$
(2.2)

where, M is the mass of a black hole and a is the rotation parameter. f(r) represents the factor term for considering dark matter. For cold dark matter (CDM) model, f(r) has the form of the following,

$$\begin{aligned} f_{\textrm{c}}(r)=\left( 1+\frac{r}{R_{\textrm{c}}}\right) ^{-\frac{8 \pi \rho _{\textrm{c}} R_{\textrm{c}}^{3}}{ r}}, \end{aligned}$$
(2.3)

and for scalar field dark matter (SFDM) model, f(r) has the form of the following,

$$\begin{aligned} f_{\textrm{s}}(r)=\exp \left( -\frac{8 \rho _{\textrm{s}} R_{\textrm{s}}^{2}}{\pi } \frac{\sin (\pi r / R_{\textrm{s}})}{\pi r / R_{\textrm{s}}}\right) , \end{aligned}$$
(2.4)

here, the parameter \(\rho \) is the density of the universe when a dark matter halo collapses, and R means its characteristic radius in this halo.

Table 1 The values of density parameters \(\rho \) and characteristic radius R from different galaxies in CDM model and SFDM model. These data are excerpted in the galaxies from Refs. [58, 59]

The corresponding dark matter parameters from the different galaxies have been fitted by the Authors in these Refs. [58, 59], and we present some of the dark matter parameters in Table 1. As can be seen from this table, these dark matter parameters correspond to specific galaxies. They are the result of a fit to the observed data. Here, we mainly take the Low Surface Brightness (LSB) galaxy ESO1200211 as our research object, and compare it with Kerr black hole.Footnote 1 The mass of black hole at the center of the LSB galaxy is approximately \(M_{\text {BH}} = 5.62 \times 10^{6}M_{\bigodot }\) [70]. For CDM model in the galaxy ESO1200211, the density parameter \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\) and the characteristic radius \(R_c=5.7\) kpc. The values of \(R_\text {c}\) and \(\rho _\text {c}\) in black hole units are \(R_\text {c}=5.7 kpc/(G \times M_{\text {BH}}/c^2)\approx 2.05 \times 10^{10} r_{\text {BH}} = 2.05 \times 10^{10}\) and \(\rho _\text {c}=2.45 \times 10^{-3}M_{\bigodot }/pc^3/(M_\text {BH}/(4/3\times \pi \times (G \times M_{\text {BH}}/c^2)^3))=3.90 \times 10^{-29} \rho _{\text {BH}}=3.90 \times 10^{-29}\times \frac{M_\text {BH}}{4/3\times \pi \times r_{BH}^3} \approx 9.33\times 10^{-30}\). For SFDM model in the galaxy ESO1200211, the density parameter \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\) and the characteristic radius \(R_s=2.92\) kpc. The values of \(R_\text {s}\) and \(\rho _\text {s}\) in black hole units are \(R_\text {s}=2.92 kpc/(G \times M_{\text {BH}}/c^2)\approx 1.05 \times 10^{10} r_{\text {BH}} = 1.05 \times 10^{10}\) and \(\rho _\text {s}=13.66\times 10^{-3}M_{\bigodot }/pc^3/(M_\text {BH}/(4/3\times \pi \times (G \times M_{\text {BH}}/c^2)^3))=2.17 \times 10^{-28} \rho _{\text {BH}}=2.17 \times 10^{-28} \times \frac{M_\text {BH}}{4/3\times \pi \times r_{BH}^3} \approx 5.20\times 10^{-29}\). In this way, all the black hole parameters in Eq. (2.1) are guaranteed to be in black hole units. On the other hand, if the impacts of dark matter on black holes is not considered, that is, \(\rho =0\), then the dark matter term will become \(f(r)=1\). The Kerr-like black hole metric in a dark matter halo will degenerate into the Kerr metric.

Fig. 1
figure 1

The functional image of \(\Delta \) as a function of independent variable r in CDM model (left panel), SFDM model and the Kerr spacetime (right panel), respectively. The function image of \(\Delta \) for different rotation parameters a are given in each panel. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

Besides, in these two black holes, the location of the event horizons can be obtained from Eq. (2.2),

$$\begin{aligned} \Delta = r^{2}f(r)-2Mr+a^{2} \approx (r-r_+)(r-r_-), \end{aligned}$$
(2.5)

where, \(r_+\) and \(r_-\) are the outer and inner horizon of this rotating black hole, respectively. Here, we also present pictures of the root of the \(\Delta \) function for these three types of event horizon in Fig. 1. We find that with the rotation parameter a increases, the roots of the \(\Delta \) function gradually change from two to one, and then to zero. This process indicates the transition from a rotation black hole to an extremal black hole. For the case of extremal black hole, it appears as the coincidence of the inner and outer horizons, and thermodynamically, it appears as Hawking temperature equals to zero [71]. Next, let’s calculate the extremal value of the rotation parameter a of the extremal black hole. Using the following ansatz, \(a^2\) can be rewritten as a function of r in black hole units \((M=1)\),

$$\begin{aligned} a^2(r)=- r^{2}f(r)+2r. \end{aligned}$$
(2.6)

According to the definition of Eq. (2.6), the coefficient of the highest order term of the independent variable r is negative. Then, the functional image of Eq. (2.6) opens downwards and then this function has a maximum value. So, the extremal value of the rotation parameter a can be transformed into the maximal value of Eq. (2.6). At this time, we only need to solve its first derivative, that is, \(d a^2(r)/dr=0\). Then, taking the maximum value r back into Eq. (2.6) to get the maximum value of \(a^2\). Finally, after square root, the extremal value of the rotation parameter a can be obtained. In black hole units, for CDM and SFDM models, the rotation parameters obtained by the numerical calculation are \(a_c \approx 1.00000021\) and \(a_s \approx 1.000000031\) with the mass \(M=1\). The rotation parameter of black hole in a dark matter halo are very close to those of Kerr black hole \((a_k=1)\), but they are larger than that of Kerr black hole. Note that Eq. (2.5) approximates the transcendental function with a quadratic function. As an example, we will discuss and analyze the possibility of this approximation mainly from f(r) and \(\Delta (r)\) with the parameters \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc in CDM model and the parameters \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc in SFDM model. Firstly, for f(r), we plot the functional image of f(r) in CDM and SFDM models in Fig. 2, as a function of r. Our results show that f(r) is almost \(f(r)=1\) over the whole space, but slightly less than 1. The biggest difference \(\delta f(r)\) between \(f_{c/s}(r)\) and \(f(r)=1\) is approximately \(1 \times 10^{-7}\). In other words, it is possible to replace this transcendental function with a quadratic function. However, to ensure the rationality of the results, we set the minimum number of digits of precision of the calculation results to 20 in the following discussion. Secondly, we set the variable \(\delta \Delta (r) = r^{2}f(r) - 2Mr + a^{2} - (r-r_+)(r-r_-)\). Then, we take the numerical results of the inner and outer horizons into \(\delta \Delta (r)\), and record the results in the following Tables 2 and 3. Our results show that within the scope of the inner and outer horizons, the maximum difference \(\delta \Delta (r)\) between the transcendental function and the quadratic function is approximately \(\delta \Delta (r)=\) \(3.73 \times 10^{-13}\), or even smaller. This difference \(\delta \Delta (r)\) relative to \(\Delta (r)\) itself is so small that it is negligible. Therefore, based on these two points, in the discussion of this work, there is some rationality when Eq. (2.5) is established.Footnote 2

Fig. 2
figure 2

The functional image of f(r) and \(\delta f(r)\) of the Kerr-like black hole in SFDM, CDM models. The main calculation parameters are \(M=1\), \(a=0.5\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc

Table 2 The variable \(\delta \Delta (r)\) with different rotation parameter a in SFDM model. The main calculation parameters are \(M=1\), \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc
Table 3 The variable \(\delta \Delta (r)\) with different rotation parameter a in CDM model. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc

Next, to make our expression more compact, we continue to use f(r) to represent the dark matter term. Then, we can obtain a covariant metric tensor \(g_{\mu \nu }\) from the Eq. (2.1),

$$\begin{aligned} g_{\mu \nu }=\left( \begin{array}{llll} -(1-\frac{r^2+2Mr-r^2 f(r)}{\Sigma ^2 }) &{}\quad 0 &{}\quad 0 &{}\quad -\frac{a \sin ^2(\theta ) \left( r^2+2Mr-r^2 f(r)\right) }{\Sigma ^2 } \\ 0 &{}\quad \frac{\Sigma ^2 }{\Delta } &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad \Sigma ^2 &{}\quad 0 \\ - \frac{a \sin ^2(\theta ) \left( r^2+2Mr-r^2 f(r)\right) }{\Sigma ^2 } &{}\quad 0 &{}\quad 0 &{}\quad \frac{A \sin ^2(\theta )}{\Sigma ^2 } \\ \end{array} \right) . \end{aligned}$$
(2.7)

With the Eq. (2.7), we can calculate the determinant of this metric,

$$\begin{aligned} g=det(g_{\mu \nu })=-\Sigma ^4\sin (\theta )^2. \end{aligned}$$
(2.8)

From Eqs. (2.7) and (2.8), we get the contravariant form of the metric,

$$\begin{aligned} g^{\mu \nu }=\left( \begin{array}{cccc} -\frac{A}{\Delta \Sigma ^2 } &{}\quad 0 &{}\quad 0 &{}\quad \frac{a r (r f(r)-2 M-r)}{\Delta \Sigma ^2 } \\ 0 &{}\quad \frac{\Delta }{\Sigma ^2 } &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad \frac{1}{\Sigma ^2 } &{}\quad 0 \\ \frac{a r (r f(r)-2 M-r)}{\Delta \Sigma ^2 } &{}\quad 0 &{}\quad 0 &{}\quad \frac{ \left( r^2 f(r)-2 M r-r^2+\Sigma ^2 \right) }{\Delta \Sigma ^2 \sin ^2(\theta ) } \\ \end{array} \right) . \nonumber \\ \end{aligned}$$
(2.9)

3 Scalar field perturbation of the Kerr-like BHs in a dark matter halo

In this section, we will study the scalar field perturbation of Kerr-like black hole and derive the radial and angular equations of scalar particles in a dark matter halo. In a curved spacetime, the equation of the motion of scalar particles can be described by the Klein-Gordon (K-G) equation,

$$\begin{aligned} \frac{1}{\sqrt{-g}}{\partial _\sigma }(\sqrt{-g}g^{\sigma \nu } {\partial _\nu }\Psi )=\mu ^{2}\Psi , \end{aligned}$$
(3.1)

where \(\mu \) is the mass of the scalar particle. With Eqs. (2.8) and (2.9), we can get the following form,

$$\begin{aligned} \begin{aligned}&\quad -\frac{A}{\Delta \Sigma ^2 }\partial ^2_t\Psi +\frac{ar(-2M-r+rf(r))}{\Delta \Sigma ^2 }\partial _t\partial _\phi \Psi \\&\quad +\frac{1}{\Sigma ^2 \sin (\theta )}\partial _\theta (\sin (\theta )\partial _\theta )\Psi +\frac{1}{\Sigma ^2 }\partial _r(\Delta \partial _r)\Psi \\&\quad +\frac{ar(-2M-r+rf(r))}{\Delta \Sigma ^2 }\partial _\phi \partial _t \Psi \\&\quad + \frac{-2Mr-r^2+\Sigma ^2+r^2f(r)}{\Delta \Sigma ^2 \sin (\theta )^2}\partial _\phi ^2\Psi =\mu ^2 \Psi . \end{aligned} \end{aligned}$$
(3.2)

Equation (3.2) is a complex second-order partial differential equation, but it can be separated by using the following ansatz,

$$\begin{aligned} \Psi (t,r,\theta ,\phi )=e^{-i\omega t}e^{im\phi }R(r)S(\theta ), \end{aligned}$$
(3.3)

where, m is the azimuthal quantum number and \(\omega \) is the frequency of this system. Therefore, we can get ordinary differential equations about angular and radial parts. If we introduce the variable \(x=\cos \theta \), with \(x\in [-1,1]\), the angular part can be written as

$$\begin{aligned} \begin{aligned}&(1-x^2)\frac{d^2S(x)}{dx^2}-2x\frac{dS(x)}{dx} \\&\quad +\left( \Lambda _{lm} + a^2k^2x^2-\frac{m^2}{1-x^2} \right) S(x)=0, \end{aligned} \end{aligned}$$
(3.4)

where, \(k^2=\omega ^2 - \mu ^2\) and \(\Lambda _{lm}\) is a separation constant. Equation (3.4) is also known as the spheroidal equation, and \(\Lambda _{lm}\) is the eigenvalue of this equation [72]. Normally, the eigenvalue \(\Lambda _{lm}\) has no analytical expression in this case. A simple method is to calculate its value by calling the Spheroidal Eigenvalue command in Mathematica. However, in the case of non-rotating limit, the spheroidal function can be reduced to a spherical harmonic function, that is, \(S_{lm} \rightarrow Y_{lm}\), and \(\Lambda _{lm}=l(l+1)\). The eigenvalue at this time can be uniquely determined, and it is related to the angular quantum number l.

Another is radial equation, and it can be written as

$$\begin{aligned} \begin{aligned}&\Delta ^2\frac{d^2R(r)}{dr^2}+\Delta \frac{d\Delta }{dr}\frac{dR(r)}{dr} \\&\quad +\left( K^2(r)-(\lambda +\mu _0^2r^2)\Delta \right) R(r)=0 \end{aligned} \end{aligned}$$
(3.5)

where, \(\Delta =(r-r_+)(r-r_-)\), \(K(r)=\omega (r^2+a^2)-am\) and \(\lambda =\Lambda _{lm}+a^2\omega ^2-2am\omega \). There are two eigenvalues \(\Lambda _{lm}, \omega \) in this equation. Among them, the eigenvalue \(\omega \) is the frequency of this system, which is a complex number. The real part of the frequency represents the oscillation frequency and its imaginary part represents the decay rate. Where, the relationship between decay rate and time scale is \(\tau ^{-1}=M \times \text {Im}(\omega )\), that is growth rate. To solve the radial equation, we need to first determine the separation constant \(\Lambda _{lm}\) in the angular equation. In the following section, instead of using the Mathematica function, we will demonstrate how to use the numerical method to determine the eigenvalues both in the angular and radial equations.

4 Continued fraction method

In this section, we will introduce the numerical method, that is, the continued fraction method. The rotating black hole is different from the general spherically symmetric black hole because there are two eigenvalues \(\Lambda _{lm}\), \(\omega \) unclear in Eqs. (3.4) and (3.5). Up to now, the continued fraction method is considered to be one of the most direct and effective methods to solve these eigenvalues. Among these two eigenvalues, \(\omega \) is the specific frequency of the black hole in Physics, and this frequency is related to the QNM/QBS [17, 27]. So, the continued fraction method also can be used to solve the QNM/QBS of rotating black holes. This method was first used by Leaver to study the QNM of the kerr black holes [26]. Next, we will strictly follow the continued fraction method used in these Refs. [26, 27, 36] and generalize it to apply to our Kerr-like black holes. Firstly, from the angular equation (3.4), we find that there are two regular singular points (\(x=-1\) and \(x=1\)) and one irregular singular point \((x=\infty )\). Therefore, their asymptotic behaviors at the location of the singular points can be written as

$$\begin{aligned} \begin{aligned} \lim _{x\rightarrow -1}S(x) \sim (1+x)^{\frac{\left| m \right| }{2}}, \quad \lim _{x\rightarrow +1}S(x) \sim (1-x)^{\frac{\left| m \right| }{2}}. \end{aligned} \end{aligned}$$
(4.1)

Taking Eq. (4.1) into account, this eigenfunction S(x) to Eq. (3.4) has the following series solution,

$$\begin{aligned} S(x)=\exp (a k x)(1-x)^{\frac{\left| m \right| }{2}}(1+x)^{\frac{\left| m \right| }{2}}\sum _{n=0}^{\infty }b_n(1+x)^n \end{aligned}$$
(4.2)

Putting Eq. (4.2) into Eq. (3.4) for calculation, it can be found that \(b_n\) must satisfy the following 3-term recurrence relation,

$$\begin{aligned} {\left\{ \begin{array}{ll} &{} \alpha _0b_1+\beta _0b_0=0, \\ &{} \alpha _nb_{n+1}+\beta _nb_n+\gamma _nb_{n-1}=0, \quad n\ge 1 \end{array}\right. } \end{aligned}$$
(4.3)

where, \(b_0=1\) and the coefficients \(\alpha _n\), \(\beta _n\), \(\gamma _n\) are as follows,

$$\begin{aligned} \begin{aligned} {\left\{ \begin{array}{ll} &{} \alpha _n= -2(n+1)(\left| m \right| +n+1),\\ &{} \beta _n=-\Lambda _{lm}+\left| m \right| (-2ak+2n+1)\\ {} &{}\qquad -ak(ak+2)+m^2+n^2-4akn+n, \\ &{} \gamma _n= 2ak(\left| m \right| +n). \end{array}\right. } \end{aligned} \end{aligned}$$
(4.4)

If we define the convergence series \(R_{n} = b_{n}/b_{n-1}\), the recurrence relation can be rewritten as

$$\begin{aligned} R_n=\frac{-\gamma _n}{\beta _n+\alpha _nR_{n+1}}. \end{aligned}$$
(4.5)

With Eq. (4.3), we also have \(R_1=b_1/b_0=-\beta _0/\alpha _0\). So, we can get the main equation of the continued fraction method,

$$\begin{aligned} \begin{aligned} 0=\beta _0-\frac{\alpha _0\gamma _1}{\beta _1-\frac{\alpha _1\gamma _2}{\beta _2-\frac{\alpha _2\gamma _3}{\beta _3-\cdots }}}. \end{aligned} \end{aligned}$$
(4.6)

Given a, k, m, these coefficients only depend on the eigenvalue \(\Lambda _{lm}\). So, the infinite continued fraction is an equation for \(\Lambda _{lm}\), and \(\Lambda _{lm}\) is a root of the continued fraction equation or any of its inversions [26, 72].

Similarly, a solution R(r) of the radial equation (3.5) can also be found since it is a spherical wave equation with similar boundary conditions as Eq. (3.4). Therefore, we can also use the continued fraction method to solve the eigenfrequency \(\omega \) in Eq. (3.5) under the help of previous discussion. On the other hand, for the perturbation theory of the black hole, the essence of the radial equation can be simplified as a wave equation. The solution to the wave equation is directly related to the location of the boundary conditions of this system, which at the event horizon and at infinity. For the behavior away from the black hole, it can be divided into two types, namely QNM and QBS. The solution of this equation is generally related to the oscillating mode of the matter field. QNM and QBS are the modes with complex frequencies. Its real part is the oscillation frequency of a black hole and the imaginary part is the decay rate of this oscillation. For the QNM, its solution is usually represented by pure incoming waves at outgoing waves at infinity. So, we require that

$$\begin{aligned} \begin{aligned}&\lim _{r\rightarrow r_+}R(r)\sim (r-r_+)^{-i\alpha }, \quad \alpha = \frac{ r_+^2+a^2}{r_+ - r_-}(\omega -m\Omega ), \\&\lim _{r\rightarrow \infty }R(r)\sim \exp (-{\bar{k}} r) r^ {\beta -1}, \quad \beta =\frac{(r_+ + r_-)(\mu ^2-2\omega ^2)}{2 {\bar{k}}}, \end{aligned} \nonumber \\ \end{aligned}$$
(4.7)

where, \(\Omega =a/(r_+ ^2+a^2)\), \({\bar{k}}=\pm \sqrt{\mu ^2-\omega ^2}\). For QNM, its behavior is pure outgoing waves far away from the black hole \((\text {Re}({\bar{k}})<0)\) and for QBS, its behavior becomes exponentially decay away from the black hole \(( \text {Im}({\bar{k}})>0)\) [27, 28].

Now, back to Eq. (3.5), there are two regular singular points (\(r=r_+\), \(r=r_-\)) and one irregular singular point (\(r=\infty \)) in this equation. Meanwhile, taking Eq. (4.7) into account, the appropriate series solution has the following form

$$\begin{aligned} \begin{aligned} R(r)&=\exp (-{\bar{k}}r)(\frac{r-r_+}{r-r_-})^{-i\alpha }(r-r_-)^{\beta -1}\\ {}&\quad \sum _{n=0}^{\infty }a_n(\frac{r-r_+}{r-r_-})^n, \end{aligned} \end{aligned}$$
(4.8)

Putting Eq. (4.8) into Eq. (3.5), it can be found that \(a_n\) must satisfy the following 3-term recurrence relation,

$$\begin{aligned} {\left\{ \begin{array}{ll} &{} \alpha _0 a_1+\beta _0 a_0=0, \\ &{} \alpha _n a_{n+1}+\beta _n a_n+\gamma _n a_{n-1}=0, \quad n\ge 1 \end{array}\right. } \end{aligned}$$
(4.9)

where, \(a_0=1\) and the coefficients \(\alpha _n\), \(\beta _n\), \(\gamma _n\) are depend on these two eigenvalues \(\Lambda _{lm}\) and \(\omega \). Since these coefficients are too complex, we do not show them here. The advantage of Eq. (4.8) is that it changes the position of the singular points in the complex plane so that, for the variable \({\bar{x}}=(r-r_+)/(r-r_-)\), the event horizon is at \({\bar{x}}=0\), infinity is at \({\bar{x}}=1\), and the other singular points are outside the unit circle. So, to find the eigenvalues \(\Lambda _{lm}\) and \(\omega \), we have to solve two continued fraction equations as before

$$\begin{aligned} \beta _0-\frac{\alpha _0\gamma _1}{\beta _1-}\frac{\alpha _1\gamma _2}{\beta _2-}\frac{\alpha _2\gamma _3}{\beta _3-} \cdot \cdot \cdot =0, \end{aligned}$$
(4.10)

Finally, this infinite continued fraction equation \(R_n\) needs to be truncated at some order \(n\in N\) to ensure the convergence of this equation [35]. Regarding the equation of convergence, Nollert had proposed some solutions to guarantee the convergence rate of this method [73, 74]. From the recurrence relation (4.9), it can be found that the convergence series \(a_{n+1}/a_n\) satisfies the following relation at the order \(n^{-1}\),

$$\begin{aligned} \frac{a_{n+1}}{a_n}= & {} 1-\sqrt{\frac{-2{\bar{k}}(r_+-r_-)}{n}}\nonumber \\{} & {} -\left[ \frac{3}{4}+\frac{(r_++r_-){\bar{k}}}{2}-2r_+{\bar{k}}+\frac{\omega ^2(r_++r_-)}{2{\bar{k}}} \right] \frac{1}{n} \nonumber \\ \end{aligned}$$
(4.11)

5 Numerical results

In this section, we will use the continued fraction method introduced in the previous section to calculate the QNM/QBS of the Kerr-like black holes immersed in a dark matter halo under the scalar field, and compare them with Kerr black hole. We will also analyze and confirm the existence of the superradiant instability of black hole, that is an unstable modes \((\text {Im}(\omega ) > 0)\). In addition, we will also study the impacts of dark matter parameters on the QNM/QBS of black holes at the specific circumstances. As previously analyzed, for the angular equation, given akm, the continued fraction equation only depend on the eigenvalue \(\Lambda _{lm}\). Similarly, for the radial equation, given \(M,a,\mu \) and m, the continued fraction equation will depend on eigenfrequency \(\omega \). In other words, the eigenvalues \(\omega \) and \(\Lambda _{lm}\) are the roots of these two continued fraction equations. Now, we successfully converted the radial/angular equations into two infinitely continued fraction equations. What needs to be done in the next step is to make a truncation at the appropriate position and ensure the convergence of the equation. So, we will truncate the continued fraction equation at the term N and use a root finding algorithm to solve the equation. Before it, one necessary step still needs to do is that giving the initial guess value of frequency \(\omega \). And the accuracy of this initial guess value will directly determine whether the results of QNM/QBS are reliable. It is seem to be a fact that we can usually equate certain physical behaviors of extremely slowly rotating black holes to the that of Schwarzschild black holes in their limit state. Witek et al. investigate this issue of the Schwarzschild backgrounds and the slow-rotating limit in their work. Their results are in agreement with the accurate values provided by the slow-rotation [75]. Besides, in Refs. [28, 53], the authors use Schwarzschild fundamental mode as their initial guess to analyze the QNM/QBS behaviors of the Kerr-like black holes. But to the behaviors of QBS, Dolan et al. have shown that, in the nonrelativistic limit, the frequency of a Schwarzschild black hole of the massive scalar field bound has the following form [27, 76],

$$\begin{aligned} \hbar \omega _n\approx \left( 1-\frac{M^2\mu ^2}{2{\bar{n}}^2} \right) \mu c^2, \end{aligned}$$
(5.1)

where, \({\bar{n}}=n+l+1\) and \({\bar{n}}\) is the principal quantum number of this system. Therefore, in this paper, we choose the Schwarzschild fundamental mode \((n=0)\) and the slowly rotating parameter \((a=0.0001)\) as our initial guess and then use a root-finding algorithm for these eigenvalues. Finally, we set the truncated term to \((N=400)\) and the minimum number of digits of precision of all the calculation results to 20. Considering the actual need (distinguishing the difference) for precision in the following analysis, we keep 9 significant figures for QNM frequencies, and 20 significant figures for QBS frequencies.

5.1 Quasinormal modes

First of all, to ensure the feasibility of our method, it is necessary for us to calculate the frequencies of fundamental quasinormal modes (QNM) for Kerr black hole with our method, and compare them with the data recorded in other published articles. Without loss of generality, we calculate the QNM frequencies of the Kerr black hole at the states of angular quantum number \(l=0\) and \(l=1\) in the massless scalar field. Note that when the angle quantum number \(l=1\), the value of the azimuthal quantum number m should be in three values \(m=-1,0,1\). Finally, we implemented the continued fraction algorithm and our results about the massless scalar field giving in Table 4 are in good agreement with the data recorded in Refs. [26, 27, 36]. Among these references, the authors all used the continued fraction method to calculate the QNM frequencies of rotating black holes. These evidences show that our method can provide an important guarantee for the calculation of the QNM frequencies in a dark matter halo.

Table 4 The fundamental quasinormal modes frequencies of the Kerr black hole for the states \(l=0\) and \(l=1\) in massless scalar field. The corresponding calculation parameter is \(M=1\)
Table 5 The fundamental quasinormal modes frequencies of the Kerr-like black hole for the states \(l=0\) and \(l=1\) in the massless scalar field with the SFDM model. The corresponding calculation parameter is \(M=1\), \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\) and \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before calculating
Table 6 The fundamental quasinormal modes frequencies of the Kerr-like black hole for the states \(l=0\) and \(l=1\) in the massless scalar field with the CDM model. The corresponding calculation parameter is \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\) and \(R_c=5.7\) kpc. We have converted these main calculation parameters to the black hole units before calculating
Fig. 3
figure 3

The fundamental quasinormal modes of the black holes in massless scalar field for the state \(l=1\) (left to right panel are the state (\(l=1, m=1\)), (\(l=1, m=0\)) and (\(l=1, m=-1\)). These points in each panel correspond to different rotation parameters a from left to right, they are \(a=0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.99\), respectively (left and middle panels), and the right panel are \(a=0.99,0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1,0\). The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

Now, let’s return to the discussion of QNM frequencies for the Kerr-like black holes in a dark matter halo. For a black hole, its oscillatory behavior is always related to dissipation, originating QNM. This oscillatory behavior usually appears as a pure incoming wave at the horizon and a pure outgoing wave at infinity. Therefore, the process of QNM is a stable mode. Here, we first study the oscillatory behavior of black holes in massless scalar field both in CDM and SFDM models, and compare them with the Kerr black hole. We use the continued fraction method to calculate these frequencies at the states of the angular quantum \(l=0, 1\) and use the Schwarzschild fundamental modes as our initial guess. We divide these QNM frequencies into two parts, real and imaginary parts. where the real part represents the oscillation frequency of this system, and the imaginary part represents the decay rate, as a function of rotating parameter a. Then, we record the QNM frequencies of the black holes about these two models at the states \(l=0\) and \(l=1\) in Tables 5 and 6, respectively. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. In Sect. 2, we have converted these dark matter parameters into black hole units, and then used them to participate in calculations. Firstly, on the whole, from Tables 45, and 6, we find that the QNM frequencies of black holes in a dark matter halo and kerr black holes at the same state are very close. In other words, the difference between the QNM frequencies of dark matter black holes and Kerr black holes is small. Taking the state \((l=m=1, a=0.99)\) as an example, both the real and imaginary parts of the QNM frequency of a Kerr black hole are greater than the SFDM model, and the SFDM model is greater than the CDM model. The QNM difference between the CDM model and Kerr spacetime is about \(1 \times 10^{-7}\) in real part and \(5 \times 10^{-8}\) in imaginary part. The difference of that between SFDM model and Kerr spacetime is about \(2 \times 10^{-8}\) in real part and \(4 \times 10^{-9}\) in imaginary part. In view of the inefficient signal-to-noise ratio of the current gravitational wave detectors [77], it may not be possible to accurately detect the frequency of dark matter and the frequency of kerr black holes. However, with the continuous update and development of detectors, future gravitational wave detectors such as LISA, Taiji [78], Tianqin [79] and DECIGO [80] may detect this kind of QNM signal. This may provide an effective method for detecting dark matter around black holes. Actually, from Table 5, we found that when the mode is \(l=1,m=-1\), the real and imaginary parts of the QNM frequencies in the SFDM model both decrease with the increasing of the rotation parameter a. However, when the modes are \(l=0,m=0\), \(l=1,m=0\) and \(l=1,m=1\), the real part of the QNM frequencies increase with the increasing of the rotation parameter a, and its imaginary part decreases with the increasing of the parameter a. This feature indicates that the large rotation parameter of the rotating black hole has a positive contribution to the QNM frequency. On the other hand, we found that when \(l=1\), the real part of the QNM frequency of a black hole in the SFDM model at the same rotation parameter a increases with the increase of m, and its imaginary part decreases with the increasing of m. Similarly, when \(m=0\), the real part of the QNM frequency in the SFDM model at the same rotation parameter a increases with the increasing of l, and its imaginary part decreases with the increasing of l. Analyzing Table 6 as we did with Table 5, we find that this trend of the oscillating behavior of black holes in the CDM model is much the same as that in the SFDM model. Finally, according to Tables 45 and 6, at the same mode (when alm are uniquely determined), we find that the oscillation frequency and decay rate of Kerr black hole are greater than that of the SFDM model, while the SFDM model is greater than the CDM model. Besides, we also graphically present these results in Fig. 3.

Fig. 4
figure 4

The real part (left) and imaginary part (right) of the fundamental quasinormal modes of the black holes in massless scalar field for the state \(l=1,m=1\). These points in each panel correspond to different rotation parameters a from left to right, they are \(a=0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.99\), respectively. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

We give the comparison charts of the QNM frequency of black holes in the Kerr spacetime, SFDM model and CDM model. We find that the QNM frequencies of black holes are very close to Kerr spacetime in dark matter models. To distinguish them, we show this difference in subfigures. The oscillation behavior of the QNM frequencies in the three states are consistent, that is, the QNM frequency of the Kerr black hole is larger than that of the SFDM model, and the SFDM model is larger than the CDM model. Finally, among the states we study, we find that the state \(l=m=1\) is quite special. The real and imaginary parts of the QNM frequencies vary widely at the different rotation parameter a. In other words, the real part of the QNM frequency increases rapidly with the increasing of the rotation parameter a, while its imaginary part decreases rapidly. But in other states, the change trend of QNM frequency is slowly increasing or decreasing. Based on this consideration, we analyze the QNM frequencies of the black hole at the state \(l=m=1\) in SFDM/CDM models, and compare them with the Kerr black hole. We give the results of our computations in Fig. 4. In order to find the difference between them, we combine black holes in SFDM model, CDM model and the Kerr spacetime in pairs and investigate the differences of QNM frequencies. The QNM frequencies of black hole both in SFDM/CDM models are lower than that of the Kerr black hole. In other words, when the mode can be in the same state (the parameters alm are determined), both the real part and the imaginary part of the QNM frequency in Kerr black hole are greater than that of the SFDM model, and the SFDM model is greater than that of the CDM model. Among them, the QNM frequency has the largest difference between the CDM model and the Kerr black hole, and the value of this difference at the same state is about \(3 \times 10^{-7}\) in real part and \(5 \times 10^{-8}\) in imaginary part. This difference of that could be that the impact of dark matter halo is too small. It is now generally believed that the dark matter distribution near black holes is a “spike” structure. This “spike” structure might amplify this difference, boosting the QNM frequency by an order of magnitude. When better-precision gravitational-wave detectors arrive, they may be able to detect this difference.

In the above we have analyzed the QNM frequencies from specific galaxy in detail. Next, we will study the impacts of dark matter parameters on the QNM frequencies both in SFDM model and CDM model. For example, in LSB galaxies, these parameters of dark matter are recorded in Table 1. In fact, Zhang et al. have investigated the impacts of dark matter parameters on the QNM frequencies in non-rotating black hole geometries [65]. However, there is evidence that the impacts of dark matter on the QNM may be enhanced in rapidly rotating black holes [81]. Therefore, here, we mainly study the QNM frequencies at the state \(l=1,a=0.99\) in a massless scalar field. Firstly, for the convenience of our analysis, for the SFDM model, we fix the density parameter \(\rho =1\times 10^{4}M_{\bigodot }/pc^{3}\) and the characteristic radius \(R=5.7kpc\) respectively. For the CDM model, we also fixed the density parameter \(\rho =1\times 10^{3}M_{\bigodot }/pc^{3}\) and the characteristic radius \(R=5.7kpc\). In other words, when the density parameter is a certain value, we calculate the QNM frequencies of the black hole, as a function of the characteristic radius R. And vice versa. Then, we calculate the QNM frequencies of black holes in the massless scalar field at state \(l=1,a=0.99\). We record our results of SFDM model and CDM model in Figs. 5 and 6, respectively.

Fig. 5
figure 5

The fundamental quasinormal modes of the black holes in massless scalar field \((\mu =0)\) for the state \(l=1,a=0.99\) in SFDM model, as a function of \(\rho \) (top) and R (bottom). The top panels we fix the \(R=5.7kpc\) and the bottom panels we fix the \(\rho =1 \times 10^{4}\) \(M_{\bigodot }/pc^{3}\). We have converted these main calculation parameters to the black hole units before plotting

Fig. 6
figure 6

The fundamental quasinormal modes of the black holes in massless scalar field \((\mu =0)\) for the state \(l=1,a=0.99\) in CDM model, as a function of \(\rho \) (top) and R (bottom). The top panels we fix the \(R=5.7kpc\) and the bottom panels we fix the \(\rho =1 \times 10^{4}\) \(M_{\bigodot }/pc^{3}\). We have converted these main calculation parameters to the black hole units before plotting

In Fig. 5, we find that when the characteristic radius is fixed, the real part of the QNM frequencies of the black holes in SFDM model decreases with the increasing of the density parameter for the state \(l=1,m=-1,0\), while their imaginary parts increase with the increasing of the density increases. At the state \(l=m=1\), the real and imaginary parts of the QNM frequencies of the black hole both decrease with the increasing of the density parameter. Similarly, when the density parameter is fixed, we investigate the impacts of the characteristic radius on the QNM frequencies of black holes. In fact, the behaviors of QNM frequencies and characteristic radius are consistent with the density parameter. This feature indicates that the dark matter around the black hole will reduce the real part of the QNM frequency of the black hole and increase the imaginary part. But from the slope of the image, the QNM frequencies seem to be more sensitive to the characteristic radius. Figure 6 is the case of the CDM model. The behavior of the CDM model is roughly the same as that of the SFDM model. The difference is that in the state \(l=m=1\), the imaginary part of QNM frequency first increases and then decreases with the characteristic radius. These results indicate that the QNM frequencies depend on the choice of dark matter parameters in these galaxies.

5.2 Quasibound states and superradiant instabilities

In the previous subsection, we considered a massless scalar field and calculated the QNM of the black holes both in CDM and SFDM models. Next, we will continue to investigate the oscillatory behavior of black holes in these two models by considering a massive scalar field, and compare them with the Kerr black hole. The oscillation behavior at this time is called quasibound states (QBS). This state often corresponds to an unusually long-lived mode [82,83,84]. Unlike QNM, QBS behaves in a form of exponential decay as it is far away from the black hole. In other words, QBS are localized inside the potential well formed by the mass of the field. On the other hand, this oscillation frequency of the QBS usually satisfies the condition of superradiant, which has the following form

$$\begin{aligned} \text {Re}(\omega ) < \mu \le m\Omega . \end{aligned}$$
(5.2)

where, \(\Omega \) is the angular velocity of rotation. Similar to QNM, the imaginary part of QBS is less than zero \(\text {Im}(\omega ) < 0\), which corresponds to a stable mode. However, for an unstable mode, it corresponds to \(\text {Im}(\omega )>0\). In this case, if it satisfy the condition of superradiance, it means superradiant instability emerging. If \(\text {Im}(\omega )=0\), it corresponds to the bound states of the so-called scalar clouds [28, 34].

Just like the previous calculation and analysis of QNM frequencies, we firstly study the QNM frequencies in CDM model, SFDM model and Kerr spacetime at the state \(l=1\) in the massive scalar field, and compare them with the massless scalar field. Here, we also use the Schwarzschild fundamental modes as our initial guess. We present our results in Fig. 7. Compared with the massless scalar field, the imaginary part of QNM frequency at the same state decreases rapidly with the increasing of mass \(\mu \) in CDM model, SFDM model and Kerr spacetime, while their real parts increase with the increasing of mass \(\mu \). This seems to be predictable, as the mass increases to a certain critical value, there will be superradiant instability, and the state \(l=m=1\) may be the fastest and most important one. Actually, in Fig. 7, we find that these QNM frequencies are still the stable modes at these states.

Fig. 7
figure 7

The fundamental quasinormal modes of the black holes in massive scalar field for the state \(l=1, m=1\) (left panel is CDM model, middle panel is SFDM model and right panel is Kerr spacetime). Three panels all reflect the stable modes. In these three pictures, three colors of red, green and blue represent three modes respectively. These points from left to right of each color are given by the increasing value of the rotation parameter a, corresponding to the values \(a=0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.99\). The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc

In order to further confirm whether these two dark matter models have superradiant instability, we use the bound state spectrum of the Schwarzschild black hole as our initial guess. At the same time, we rewrite the oscillation frequency and decay rate as \(\text {Re}(\omega )/\mu \) and \(\text {Im}(\omega )/\mu \), respectively, as a function of the mass parameter \(M\mu \). Then, we investigate all the state for the angular quantum \(l=1\) in the case of the nearly extremal parameter (\(a=0.99\)) in CDM model, SFDM model and Kerr spacetime. We show our calculation results in Fig. 8.

Fig. 8
figure 8

The quasibound states of the black holes in massive scalar field for the state \(l=1\) (left panel is CDM model, middle panel is SFDM model and the right panel is Kerr spacetime). Subfigures in three panels all reflect the superradiant instabilities. The maximum instability occurs for the state (\(l=1, m=1\)). The main calculation parameters are \(a=0.99\), \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

From Fig. 8, we found that the imaginary part of the QBS in these three panels are negative numbers at the range of large mass, which indicates that QBS is a stable mode in the large mass. However, at the range of the low mass, the value of the decay rate \(\text {lm}(\omega )/\mu \) is very close to 0 (may be greater than 0 or less than 0), and the former case may lead to the occurrence of superradiant instability. So, in the subfigures of each panel, we zoomed in the QBS frequencies in the low mass range. These subfigures all demonstrate superradiant instabilities. Among the three state \((l=1,m=1), (l=1,m=1)\) and \((l=1,m=1)\), the maximum instability of the black hole appears at the state \(l=1, m=1\). Up to here, our results show that there are instabilities both in SFDM/CDM models. This appears to be due to bound state spectra. This instability seems to be sensitive to the initial guess value. Actually, this bound state spectrum plays an important role in analyzing QBS.

Fig. 9
figure 9

The quasibound states of the black holes in massive scalar field for the state \(l=1, m=1\) (left panel is CDM model, middle panel is SFDM model and the right panel is Kerr spacetime). Subfigures in three panels all reflect the superradiant instabilities, and the maximum instabilities all occur approximately for the state \(a=0.995\). The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

In the above, our results show the existence of superradiant instabilities both in the SFDM model and the CDM model. Next, we will qualitatively analyze and demonstrate in which state the maximum instability occurs. Firstly, based on the instabilities at the state \(l=1\), what we need in the following is that determining the rotation parameter a in the maximum instabilities. By the fixed rotation parameter a, we show the instabilities as the function of mass \(M\mu \) in Fig. 9. The rotation parameter a is \(0\le a \le 0.9995 (a_{max} \approx 1)\). From these subfigures, for each rotation parameter a, there is always a maximum value of the instability. For these subfigures in the top three panels, the maximum value of these maximum instabilities increases with the increasing of the rotation parameter a. But in three panels of the bottom, the maximum value of instabilities first increases and then decreases with the increasing of the rotation parameter a. This turning point is rotation parameter \(a= 0.995\). These results indicate that there is a maximum value of instability when the rotation parameter \(0 \le a \le 0.9995 (a_{max} \approx 1)\), i.e., this turning point \(a= 0.995\). Therefore, at the state \(l=1\), the maximum instability occurs approximately for the state \(l=1, m=1, a=0.995\). Secondly, we need to determine what the maximum instabilities occur in at the states with different l.

Fig. 10
figure 10

The quasibound states of the black holes in massive scalar field at the state \(l=m=1, 2, 3, a=0.995\) (left panel is CDM model, middle panel is SFDM model and the right panel is Kerr spacetime). Three panels all reflect the superradiant instabilities, and the maximum instabilities all occur for the state \(l=m=1\). The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc. We have converted these main calculation parameters to the black hole units before plotting

In Fig. 10, we show the instabilities at the states \(l=m=1, 2, 3\). The maximum instability decreases both with the increasing of lm. So, the maximum instability occur for the state \(l=m=1\). Finally, based on the above discussions, we prove that in CDM model, SFDM model and Kerr spacetime, the maximum instability occurs approximately for the state \(l=1, m=1, a=0.995\). This result was as previously expected. In order to further study and analyze the difference in the maximum instability between dark matter black holes and Kerr black holes, in Fig. 11, we show the instabilities at the state \(l=1, m=1, a=0.995\) in the CDM model, the SFDM model and the Kerr spacetime. The maximum instabilities of the Kerr black hole is larger than that of the SFDM model, and the SFDM model is larger than the CDM model. After comparing Kerr black holes both with the SFDM model and CDM model, the difference of the maximum instability between the black hole in CDM model and the Kerr black hole is approximately \(1.4\times 10^{-10}\) with \(M\mu =0.5\). The difference of the maximum instability between Kerr black hole and black hole in the SFDM model is approximately \(2.0\times 10^{-11}\) with \(M\mu =0.5\). These differences increase with the increasing of mass. This result directly reflects the impacts of the mass on the superradiant mechanism. With the increasing of the mass, that is, when the mass satisfies \(\text {Re}(\omega )<\mu \), this mass can work as a mirror [32, 84, 85]. This causes the wave to bounce back and forth between the black hole and the mirror, and it can amplify itself each time. This is the so-called the black hole bomb.

Fig. 11
figure 11

The superradiant instabilities of the black holes in massive scalar field at the state (\(l=1, m=1, a=0.995\)) in CDM model, SFDM model and Kerr spacetime. The instability of the Kerr black hole is larger than that of the SFDM model, and the SFDM model is larger than the CDM model (left panel). After comparing Kerr black holes both with the SFDM model and CDM model, the real part of the comparing results are shown in the middle panel and the imaginary part are shown in the right panel. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc

According to the above analysis, we have proved that the maximum instability of QBS occurs approximately for the state \(l=1, m=1, a=0.995\) in the SFDM model, CDM model and Kerr spacetime. Next, we will quantitatively analyze the maximum instability at the state \(l=1, m=1, a=0.995\). Here, we show that how to find these maximum instabilities of the state \(l=1, m=1, a=0.995\) in the CDM model, SFDM model and Kerr spacetime. First of all, through the results in Fig. 9, we can roughly determine a range of mass corresponding to the maximum instability, and form an array of the mass \(M\mu \) and decay rate \(\text {Im}(\omega )\). Secondly, we reduce the mass stepwise and use the continued fraction method to determine the frequency of each step. where the initial guess of frequency is the frequency obtained in the previous step. Finally, we stop iterating as soon as a local maximum occurs among a series of frequencies. Following this procedure strictly, we obtain the maximum instability of black holes for the state \(l=1, m=1,a=0.995\) in the CDM model, SFDM model and Kerr spacetime. We show these results in Fig. 12.

Fig. 12
figure 12

The superradiant instabilities of the black holes for different rotation parameter a in massive scalar field at the state (\(l=1, m=1\)) in CDM model (left panel), SFDM model and Kerr spacetime (right panel). In the three panels of the top, the maximum instabilities all increase with the increasing of the rotation parameter a. But in three panels of the bottom, the maximum instabilities first increase and then decrease with the increasing of the rotation parameter a. This turning point is \(a=0.995\). The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc, \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc

From these figures, the maximum instabilities increase with the increasing of the rotation parameter a in the three panels of the top. But in three panels of the bottom, the maximum instabilities first increase and then decrease with the increasing of the rotation parameter a. This turning point is \(a = 0.995\). Finally, by the rotation parameter a at this state, we quickly obtain the value of mass \(\mu \). Since these results for black holes in CDM/SFDM models are very close to Kerr black hole, we give their numerical results of the maximum instabilities growth rate (\(\tau ^{-1}=M\text {Im}(\omega )\)) in Tables 78 and 9.

Table 7 The maximum instability growth rate with different rotation parameter a at the state \(l=m=1\) in Kerr black hole. And the maximum instability growth rate (\(\tau ^{-1}=M\text {Im}(\omega )\)) and the corresponding mass \(\mu \) are given. The \(\Delta \) is the relative error relative to the Table III in Ref. [27]
Table 8 The maximum instability growth rate with different rotation parameter a at the state \(l=m=1\) in SFDM model. The maximum instability growth rate (\(\tau ^{-1}=M\text {Im}(\omega )\)) and the mass \(\mu \) are given. The main calculation parameters are \(M=1\), \(\rho _s=13.66 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_s=2.92\) kpc
Table 9 The maximum instability growth rate with different rotation parameter a at the state \(l=m=1\) in CDM model. The maximum instability growth rate (\(\tau ^{-1}=M\text {Im}(\omega )\)) and the corresponding mass \(\mu \) are given. The main calculation parameters are \(M=1\), \(\rho _c=2.45 \times 10^{-3}\) \(M_{\bigodot }/pc^{3}\), \(R_c=5.7\) kpc
Fig. 13
figure 13

The quasibound states of the black holes in massive scalar field \((\mu =0.440)\) at the state \(l=1, m=1, a=0.995\) in SFDM model, as a function of \(\rho \) (top) and R (bottom). The top panels we fix the \(R=5.7kpc\) and the bottom panels we fix the \(\rho =1 \times 10^{3}\) \(M_{\bigodot }/pc^{3}\). The right panels reveal the instabilities. We have converted these main calculation parameters to the black hole units before plotting

We list the maximum instability growth rate of CDM model, SFDM model and Kerr spacetime with different rotation parameters a at the state \(l=1, m=1\) in Tables  78 and 9, respectively. Besides, in Table 7, we also give the relative error between the maximum instability and the test value from Ref. [27] in the last row. These results demonstrate that our approach is reliable. From the data from Tables 7, 8 and 9, both in CDM and SFDM models, the maximum instability increases with the increasing of the rotation parameter a. For CDM model, the maximum instability occurs approximately for \(M\mu \approx 0.440 \) and the maximum instability growth rate is approximately \(\tau ^{-1} =M\text {Im}(\omega ) \approx 1.69918377 \times 10^{-7} (GM/c^3)^{-1}\). For SFDM model, the maximum instability occurs approximately for \(M\mu \approx 0.440 \) and the maximum instability growth rate is approximately \(\tau ^{-1} =M\text {Im}(\omega ) \approx 1.69918500 \times 10^{-7} (GM/c^3)^{-1}\). The difference between them is approximately \(1.2\times 10^{-13} (GM/c^3)^{-1}\). Compared the maximum instabilities of black holes in CDM/SFDM models with Kerr black hole, the values of the maximum instability of Kerr-like black holes in a dark matter halo are all smaller than that of Kerr black hole. The value of the maximum instability of the Kerr black hole is larger than that of the SFDM model, and the SFDM model is larger than that of the CDM model. The maximum instability difference of black hole between CDM model and Kerr black hole is approximately \(1.4\times 10^{-13}\). For SFDM model, the difference is approximately \(2.0 \times 10^{-14}\). These values are an upper bound on the growth rate of instabilities of black holes in massive scalar field in Tables 78 and 9. These results show that, in the three cases considered, the growth rate of the Kerr black hole instability is larger than that of the SFDM model, which is larger than that of the CDM model.

Finally, we also study the impacts of dark matter parameters on the QBS frequencies, just like the previous analysis of how the dark matter parameters affect the QNM frequency. Here, we mainly consider to test the QBS frequencies at the state \(l=m=1,a=0.995\) in a massive field (\(\mu \ne 0\)). Similarly, we need a set of fixed dark matter parameters for analysis. For the SFDM model, we fix the density parameter \(\rho =10^{4}M_{\bigodot }/pc^{3}\) and the characteristic radius \(R=5.7kpc\) respectively. For the CDM model, we also fixed the density parameter \(\rho =10^{3}M_{\bigodot }/pc^{3}\) and the characteristic radius \(R=5.7kpc\). Then, we calculate the QBS frequencies of black holes in the massive scalar field \((\mu =0.440)\) at state \(l=m=1,a=0.995\). We record our results of SFDM model and CDM model in Figs. 13 and 14, respectively. In Fig. 13, we find that when the characteristic radius is fixed, the real and imaginary parts of the QBS frequencies of the black hole both decrease with the increasing of the density parameter. Similarly, when the density parameter is fixed, we investigate the impacts of the characteristic radius on the QBS frequencies of black holes. Actually, the behaviors of QBS frequencies and characteristic radius are consistent with the density parameter. These features indicates that the dark matter around the black hole will reduce the real and imaginary parts of the QBS frequencies. Figure 14 is the case of the CDM model. The behavior of the CDM model is roughly the same as that of the SFDM model.

Fig. 14
figure 14

The quasibound states of the black holes in massive scalar field \((\mu =0.440)\) at the state \(l=1, m=1, a=0.995\) in CDM model, as a function of \(\rho \) (top) and R (bottom). The top panels we fix the \(R=5.7kpc\) and the bottom panels we fix the \(\rho =1 \times 10^{3}\) \(M_{\bigodot }/pc^{3}\). The right panels reveal the instabilities. We have converted these main calculation parameters to the black hole units before plotting

In addition, we have also calculated the superradiant instabilities of the black holes in the massive scalar field at the state \(l=m=1,a=0.995\) in the galaxies numbered ESO from the Table 1. We show the results in Fig. 15. We find that the real part of the QBS frequencies of these galaxies increases with the increasing of mass, while the imaginary part increases first and then decreases.

Fig. 15
figure 15

The quasibound states of the black holes from different galaxies in massive scalar field at the state (\(l=1, m=1, a=0.995\)) in the CDM model (left panel) and SFDM model (right panel). These two panels show superradiant instabilities. The maximum instability of Kerr black holes is larger than that of other galaxies. The points in each panel correspond to different mass, from left to right are the \(\mu =0.2\) to \(\mu =0.55\), with a step-size of 0.01. The main dark matter parameters are from the data in Table 1

Our results show that these dark matter parameter configurations can all lead to superradiant instabilities both in SFDM and CDM models. Among them, the maximum instabilities both in SFDM model and CDM model are lower than that of Kerr black holes. In other words, for nearly extremal black holes, the distribution of dark matter reduces the imaginary part of the QBS frequency compared to Kerr black holes.

6 Conclusions and discussions

In this paper, we mainly use the scalar field perturbation to study QNM/QBS of the Kerr-like black holes in a dark matter halo, and compare these results with Kerr black hole. The method we used for calculating the frequencies of QNM/QBS is the continued fraction method. In addition, we also study the impacts of dark matter parameters on the QNM/QBS of black holes at the specific circumstances. Our main conclusions are as follows:

(1) In the massless scalar field, the real and imaginary parts of QNM frequencies of the black hole at the state \(l=1\) both increase with the increasing of m in CDM model, SFDM model and Kerr spacetime. For the same state, QNM frequency of kerr black hole is greater than that of SFDM model, and SFDM model is greater than that of CDM model. For the state \(l=m=1,a=0.99\), the QNM difference between the CDM model and Kerr spacetime is approximately \(1 \times 10^{-7}\) in real part and \(5 \times 10^{-8}\) in imaginary part. The QNM difference of that between SFDM model and Kerr spacetime is approximately \(2 \times 10^{-8}\) in real part and \(4 \times 10^{-9}\) in imaginary part.

(2) In the massive scalar field, by testing the states of QBS frequencies with different lma, we confirm the existence of the superradiant instabilities when the black holes both in CDM and SFDM models. Besides, we prove that the maximum instabilities of black hole in CDM model and SFDM model occur approximately for the state \(l=1,m=1, a=0.995\).

(3) In black hole units, for CDM model in ESO1200211, the superradiant instability of black hole occurs approximately for the mass parameter \(M\mu \lesssim 0.55\). Its maximum instability occurs approximately for \(M\mu \approx 0.44 \) and the maximum instability growth rate is approximately \(\tau ^{-1} \approx 1.69918377 \times 10^{-7} (GM/c^3)^{-1}\). For SFDM model in ESO1200211, the superradiant instability of black hole occurs approximately for mass parameter \(M\mu \lesssim 0.55\). Its maximum instability occurs approximately for \(M\mu \approx 0.44 \) and the maximum instability growth rate is approximately \(\tau ^{-1} \approx 1.69918500 \times 10^{-7} (GM/c^3)^{-1}\). These values are the upper bounds on the instability growth rate of black holes in the massive scalar field. The growth rate of the maximum instability of black hole in SFDM model is greater than that of CDM model. The difference between them is approximately \(1.2\times 10^{-13} (GM/c^3)^{-1}\).

(4) Compared the maximum instabilities of black holes in CDM/SFDM models with Kerr black hole, the values of the maximum instability of Kerr-like black holes in a dark matter halo are all smaller than that of Kerr black hole. The maximum instability difference of black hole between CDM model and Kerr black hole is approximately \(1.4\times 10^{-13}\). For SFDM model, the difference of that is approximately \(2.0\times 10^{-14}\). In the future, these differences may be detected by the gravitational wave detection, which may provide an effective method for detecting the existence of dark matter.

(5) The impacts of dark matter parameters on the QNM/QBS of black holes at the specific circumstances are studied. The dark matter parameters affecting QNM/QBS are density parameter \(\rho \) and characteristic radius R. The method of the research is the control variable method. For QNM frequencies both in SFDM model and CDM model, at the state \(l=1,m=-1,0\), the real part of QNM frequencies decrease with the increasing of dark matter parameter, and their imaginary parts increase with the increasing of dark matter parameter. At the state \(l=m=1\), the real and imaginary parts of the QNM frequencies decrease with the increasing of the dark matter parameter. For QBS frequencies both in SFDM model and CDM model, at the state \(l=1,m=1,a=0.995\), the real and imaginary parts of the QBS frequencies decrease with the increasing of the dark matter parameter.

At last, it is also worth mentioning that we are very interested in the exploration of the relevant physical processes of black holes around the dark matter. Based on this premise, we choose to study the QNM/QBS of a rotating black hole in a dark matter halo. In fact, the distribution of dark matter around a black hole is a “spike” structure, and this structure is closer to the real situation. The QNM/QBS of a special black hole are the characteristic “sound” which can provide us with a new method to identify black holes in the universe. On the other hand, in some recent studies, we found some discussion about echoes [86,87,88,89,90,91]. These studies show that the echoes appear after the quasinormal mode. Echoes are one of the important means currently used to test gravitational waves. Taking the above two points into consideration, our next plans are to study the physics of rotating black holes associated with echoes. About the study of echoes, we have already made some preliminary attempts [92,93,94]. To sum up, we also hope that our work can form a complete research system in the direction of interaction between dark matter and black holes. In the future, these studies may be used for gravitational wave detection of supermassive black holes, and may provide an effective method for detecting the existence of dark matter.