Efficient spectroscopy of exoplanets at small angular separations with vortex fiber nulling

Instrumentation designed to characterize potentially habitable planets may combine adaptive optics and high-resolution spectroscopy techniques to achieve the highest possible sensitivity to spectral signs of life. Detecting the weak signal from a planet containing biomarkers will require exquisite control of the optical wavefront to maximize the planet signal and significantly reduce unwanted starlight. We present an optical technique, known as vortex fiber nulling (VFN), that allows polychromatic light from faint planets at extremely small separations from their host stars ($\lesssim\lambda/D$) to be efficiently routed to a diffraction-limited spectrograph via a single-mode optical fiber, while light from the star is prevented from entering the spectrograph. VFN takes advantage of the spatial selectivity of a single-mode fiber to isolate the light from close-in companions in a small field of view around the star. We provide theoretical performance predictions of a conceptual design and show that VFN may be utilized to characterize planets detected by radial velocity (RV) instruments in the infrared without knowledge of the azimuthal orientation of their orbits. Using a spectral template-matching technique, we calculate an integration time of $\sim$400, $\sim$100, and $\sim$30 hr for Ross 128 b with Keck, the Thirty Meter Telescope (TMT), and the Large Ultraviolet/Optical/Infrared (LUVOIR) Surveyor, respectively.


INTRODUCTION
Perhaps the only practical pathway for detecting biosignatures with ground-based telescopes is to obtain high-resolution spectra of planets orbiting the nearest M-dwarf stars (Riaud & Schneider 2007;Snellen et al. 2015;Wang et al. 2017). The recent discoveries of Proxima Centauri b (Anglada-Escudé et al. 2016) and Ross 128 b (Bonfils et al. 2017) are examples of what may be a plentiful sample of tantalizing targets. However, these planets are out of reach for current high-contrast imaging instruments because of the extremely small angular separation (<37 and <15 mas, respectively) and flux ratio (∼ 10 −8 -10 −7 ) between the planets and their host stars. The necessary inner Corresponding author: Garreth Ruane gruane@caltech.edu * NSF Astronomy and Astrophysics Postdoctoral Fellow working angle in both cases is smaller than the angular resolution of a 10 meter telescope in the infrared. Moreover, the wavefront control precision needed to sufficiently suppress unwanted starlight is ∼100× better than provided by current state-of-the-art adaptive optics (AO) systems. The discovery of life on these worlds via imaging spectroscopy may therefore need to wait for next-generation extreme AO on a giant segmented mirror ground-based telescopes, such as the Planetary Systems Imager (PSI) on the Thirty Meter Telescope (TMT), or large-aperture space telescopes such as the Large Ultraviolet/Optical/Infrared Surveyor (LUVOIR; Bolcar et al. 2016;Pueyo et al. 2017). Even with a primary mirror diameter of TMT (D=30 m), this application requires an instrument that provides robust rejection of light from the star whose photon noise contribution overwhelms the relatively few photons from a planet with an angular separation of λ/D (Kawahara et al. 2014).

Vortex mask
Lyot stop (in pupil) DM Single mode fiber Figure 1. Schematic of the optical system. The wavefront from the telescope is flattened by a deformable mirror (DM) and the beam is focused such that the star is aligned at the center of the vortex phase mask in the focal plane. Some of the starlight is blocked at the Lyot stop in the downstream pupil. The single mode fiber (SMF) is centered at the geometric image of the star and is 2.5× larger than λF # at the fiber to simultaneously capture off-axis planet light. Since the vortex mask imparts an azimuthal phase ramp given by exp(±iθ), the reimaged stellar field is orthogonal to the fundamental mode of the SMF and is therefore rejected.
Here, we present the vortex fiber-nulling (VFN) concept, which imparts an optical vortex phase pattern on the focused starlight (Foo et al. 2005;Mawet et al. 2005) causing it to be rejected by a single-mode fiber (SMF) in the subsequent focal plane in a process akin to fiber-nulling interferometry Mennesson et al. 2006;Por & Haffert 2018). The SMF feeds light from planets at angular separations ∼ λ/D into a high-resolution spectrograph (R = λ/∆λ ≈ 100,000). The spectrum is then analyzed using high-dispersion coronagraphy methods (Sparks & Ford 2002;Riaud & Schneider 2007;Snellen et al. 2015;Wang et al. 2017;Mawet et al. 2017a;Lovis et al. 2017) to reveal close-in planetary mass companions in small field of view around the star and identify molecular species in their atmospheres.
Our optical design may potentially be used to detect and characterize planets as close as ∼0.4 λ/D from the star over a wide spectral range and does not require precise prior knowledge of the orientation of the planet's orbit. The VFN method will unlock the potential to discover and characterize low-mass exoplanets in the near future at the W.M. Keck Observatory with minor modifications to the upcoming Keck Planet Imager and Characterizer (KPIC) instrument (Mawet et al. 2016;Mawet et al. 2017b). Using Ross 128 b as an example, we compute the integration time needed to detect potential signs of life in the atmospheres of terrestrial planets orbiting in the habitable zone of nearby M stars with Keck, TMT, and LUVOIR. The feasibility of such observations is dependent on the AO system's ability to control a select few low-order wavefront error modes, namely tip-tilt and coma, and is relatively insensitive to mid and high spatial frequency aberrations.

VFN CONCEPT
Vortex fiber nullers are designed to be sensitive to planet light at small angular separations while suppressing unwanted starlight. Figure 1 shows a schematic of a VFN instrument consisting of a deformable mirror (DM) for wavefront control, followed by a vortex phase mask in the focal plane with complex transmittance exp(±iθ), where θ is the polar angle in the focal plane. The starlight is centered on the phase singularity of the focal plane mask. A VFN instrument is different than a vortex coronagraph in that the starlight is then re-imaged onto a co-aligned SMF, which rejects it. That is, the coupling efficiency of the field, f (r, θ), at the fiber tip is nulled if the overlap integral with the fiber mode is zero; i.e., where ψ(r) is the fundamental mode of the SMF. For common SMFs, the mode may be approximated by , where D f is the mode field diameter. The starlight has an azimuthally varying phase term of the form exp(±iθ), which is orthogonal to the uniform phase of ψ(r) and therefore prevents the starlight from coupling into the SMF. In fact, Eqn. 1 is zero if the stellar field is of the form f (r, θ) = f r (r) exp(ilθ) with a nonzero integer value of l. Furthermore, we confirmed numerically that an azimuthally dependent fiber mode, ψ(r, θ), may also reject the exp(ilθ) term for a variety of fold symmetries, including the hexagonal mode of a photonic crystal fiber (Birks et al. 1997).
Assuming that there is no wavefront error in the system, light from an on-axis point source does not enter the SMF regardless of the F # of the focusing optic. We therefore align the SMF to the position of the star and choose the mode field diameter and focusing optic such that D f = 2.5λF # to maximize planet coupling effi- ciency. In this configuration, the starlight is rejected by the SMF, and the light from planets at small angular separations ( λ/D) is partially coupled. Figure 2 illustrates the SMF spatial filtering mechanism with Keck's segmented, noncircular, obstructed aperture (Fig. 2a). The Lyot stop (Fig. 2b) is designed to only block starlight that is diffracted outside of the geometric pupil in order to reduce the amount of starlight in the image plane without paying a significant throughput penalty. The azimuthal phase dependence of the stellar field at the fiber tip ( Fig. 2c) causes Eqn. 1 to compute to zero and therefore no starlight can propagate into the SMF. On the other hand, ∼20% of the light from a planet at an angular separation of ∼0.9λ/D (Fig. 2d) makes its way into the fiber and transmits to the spectrograph.
Whereas traditional fiber-nulling interferometers create destructive interference between sub-apertures , VFN has the advantage of using the entirety of the telescope aperture and does not require baseline modulation (Mennesson et al. 2006) to detect sources at all possible azimuthal orientations. The VFN starlight cancellation mechanism also does not depend on the wavelength nor the shape of the pupil. The coupling efficiency of the stellar beam is zero regardless of the shape of the Lyot stop; in fact, the Lyot stop may be removed from the system. We have opted to include a Lyot stop because it offers the practical benefit of preventing 70% of the starlight from . Theoretical planet throughput into an SMF, located on the optical axis with a mode field diameter of D f = 2.5 λF # , downstream of a vortex phase mask with complex transmittance exp(±iθ). This calculation accounts for diffractive losses owing to the phase mask and Lyot stop as well as the SMF coupling efficiency. With the star centered on the optical axis, light from point sources in an annular region around the star will transmit into the spectrograph. The inset shows the 2D throughput map for planets in a 4×4 λ/D field of view. The throughput only changes by 1-2% for a range of possible telescope apertures.
reaching the image plane, assuming a flat wavefront, while only reducing the planet throughput by 5%. The fraction of the planet light that couples into the SMF, η p , is shown in Fig. 3. Using the on-axis fiber mitigates the need to know the planet's precise position in advance, since the throughput does not depend on azimuth. With the star and SMF co-aligned, light from point sources in an annular region around the star will couple into the SMF (see Fig. 3, inset). However, the mode entering the spectrograph will be ψ(r) regardless of the planet location, enabling temporally stable and diffraction-limited spectroscopy.

THEORETICAL PERFORMANCE
In this section, we predict the performance of VFN in realistic noise environments. We write the signal from the planet and star that enters the spectrograph as S p = η p Φ p τ ∆λAqT and S s = η s Φ s τ ∆λAqT , where η p and η s are the planet and star throughputs of the VFN, Φ p and Φ s are the flux owing to the planet and star (photons per unit area per unit time per unit wavelength at the primary mirror), τ is the integration time, ∆λ is the full spectral bandwidth, A is the collecting area of the telescope, q is the quantum efficiency of the detector, and T is the transmission of the instrument describing losses that affect the star and planet equally. When limited by stellar photon noise, the signal-tonoise ratio (SNR) per spectral channel is where λ is the wavelength and R is the spectral resolution. Solving for the integration time to achieve a given SNR: where = Φ p /Φ s is the flux ratio between the planet and the star and The objective of VFN is to minimize the integration time to detect molecules in a planet's atmosphere. In the remainder of this section, we confirm that the VFN approach offers significant reductions in integration time when tip-tilt and coma wavefront errors are sufficiently controlled. We also discuss the effects of partially resolved stars, background noise, and detector noise.

Tip-tilt jitter
On ground-based telescopes, the performance of the proposed system will likely be limited by the ability of current AO systems to accurately sense and correct tiptilt errors at kHz rates. We assume in the following that atmospheric dispersion is compensated to high precision (e.g. using methods demonstrated in Pathak et al. 2018) and is therefore negligible. Figure 4 shows η p and η s as a function of tip-tilt jitter for various angular separations of the planet. The planet throughput (Fig. 4a) is relatively insensitive to tip-tilt jitter <0.1 λ/D rms. On the other hand, the fraction of starlight that leaks into the SMF in the VFN configuration ( Fig. 4b) may be approximated as where σ is the rms tip-tilt jitter in radians for σ 1 rad. The relative integration time ( Fig. 4c) is given by τ /τ 0 = η s /η 2 p . The VFN configuration significantly reduces integration time with rms tip-tilt jitter that is λ/D.

Angular size of the star
The fundamental lower limit of η s is reached when the tip-tilt errors become significantly smaller than the angular size of the star. We treat the star as a ensemble of incoherent point sources in a uniform disk. For unresolved sources, η s may be approximated as variance of the source distribution in units of λ/D (Ruane 2016): where Θ is the angular diameter of the source. The minimum possible integration time is therefore More generally, η s may be approximated by summing the variances of the source and jitter distributions.

Low-order aberrations in the Zernike basis
A major benefit of VFN is that it does not rely on high-precision control of all low-order aberrations. Although an overall rms wavefront error of ω < 1/10 waves Coupled modes Rejected modes Figure 5. Amplitude (value) and phase (hue) of the stellar field in the central 4×4 λF # region with λ/10 rms wavefront error in each of the lowest 11 Zernike modes, Z m n (r, θ) (excluding piston, which is shown in Fig. 2). The white circle indicates the mode field diameter D f = 2.5 λF # . Modes with |m| = 1 leak starlight with ηs ∝ ω 2 whereas those with |m| = 1 do not couple into the SMF; i.e. ηs = 0.
is desired for optimal planet light coupling, the nulling of the star is unaffected by Zernike aberrations, Z m n (r, θ), if |m| = 1. To provide a qualitative explanation, we write the aberrated stellar field at the pupil as a linear combination of functions of the form R m n (r)e ±imθ , where R m n (r) are the radial Zernike polynomials. Since the exp(±iθ) pattern imparted on the stellar beam acts to increase the effective azimuthal order of the beam by ±1, the starlight will only couple into the SMF if the the azimuthal order of the Zernike polynomial is canceled; that is, if l + m = 0 or l − m = 0. Thus, stellar leakage is only induced by tip-tilt and radial orders of coma, where m = ±1.
For small aberrations, the amount of leaked starlight is where b nm is the sensitivity constant. Table 1 lists numerically determined b nm values for the 8 lowest critical modes (m = ±1). For all other modes where |m| = 1, b nm = 0 and η s = 0. Figure 5 illustrates this effect by plotting the phase of the field at the SMF due to an onaxis point source. The coupling is zero when the phase has a azimuthal phase ramp and/or fold symmetry. The integration time needed to overcome the noise from critical low order aberrations may be approximated by where τ nm is the integration time for each Zernike aberration. Figure 6a shows τ /τ 0 as a function of ω for the lowest order critical modes using the power-law approximation (solid lines) as well as full numerical beam propagation (dotted lines). As expected, τ has quadratic dependence on ω for ω 1 wave, though the full simulation deviates from this behavior in some cases. From an instrument design point of view, it is useful to budget integration time costs for dominant low order aberrations by writing the wavefront error requirements as For example, in order to achieve τ /τ 0 = 10 −3 (i.e. with jitter 10 −2 λ/D rms), the wavefront error requirements in the first three orders of coma (Z ±1 3 , Z ±1 5 , and Z ±1 7 ) would be λ/450, λ/40, and λ/5 rms. Since most of the power in a typical aberrated wavefront is contained in the lowest order modes, a VFN instrument should be tailored to sense and correct tip-tilt and primary coma (Z ±1 3 ) to the highest possible precision. Current AO systems provide adequate control of all other modes.

Aberrations in the Fourier basis
Although it is sufficient to describe the wavefront with Zernike polynomials, we also report the sensitivity to aberrations in the Fourier basis. By symmetry arguments similar to the case of low-order aberrations with |m| = 1, the VFN approach is completely robust to even Fourier modes (cosines). It is, however, sensitive to odd Fourier modes (sines) with low spatial frequency. Again, we describe the amount of leaked starlight as where b ξ are constants that we determined numerically (see Table 1). Figure 6b shows the relative integration time τ F /τ 0 = η s /η 2 p owing to Fourier modes as a function of ω for spatial frequencies ξ = 1, 2, and 3 cycles per pupil diameter, confirming the quadratic dependence in the small error regime (solid lines). Our full simulations (dotted lines) deviate from this power law for ω 1/10 waves rms mainly because of a significant drop in planet coupling efficiency.
The VFN approach is not effected by mid spatial frequency aberrations, which generate the speckle noise that tends to limit the performance of high-contrast imagers. In other words, VFN is relatively insensitive to Fourier modes with spatial frequencies >2-3 cycles per pupil diameter. It is therefore only necessary to control mid to high spatial frequencies to ∼ λ/10 rms in order to achieve close to ideal planet coupling.
Since VFN is insensitive to high spatial frequency aberrations, the primary mirror co-phasing requirements are also somewhat relaxed for segmented telescopes. To demonstrate this, we simulated a segmented mirror with Gaussian random segment piston and tiptilt errors (see Fig. 6c). Averaging over many realizations of the primary mirror, we find that VFN is most sensitive to random piston errors and is more robust to random tilts between segments. Compared to the wavefront error requirement for the primary coma term (Z ±1 3 ), the requirement for random segment pistons is relaxed by a factor of ∼10 for the same integration time.

Other noise sources
For completeness, we include a similar analysis to determine the effect of sky and telescope thermal emission and detector noise. For thermal background noise, the integration time is given by where L bg is the background radiance and Ω fib is the solid angle subtended by the fiber. For the detector dark current, i d , the integration time is The read noise over long integration observations has an approximate variance of σ 2 rd = N 2 rd N frames ≈ N 2 rd S s /W , where N rd is the read noise, N frames is the number of frames, and W is the full well depth. Thus, the integration time to overcome read noise is which depends on η s and, by extension, on the estimated optical aberrations present.

Estimating total integration time
Since the variance of each noise term has been approximated to be linear with integration time, the total integration time may be estimated by summing the integration time to overcome each independent noise source: which is equivalent to summing the individual noise variances. With this expression, it is straightforward to In this section, we investigate the feasibility of detecting and characterizing the recently discovered planet Ross 128 b with a VFN instrument on Keck, TMT, and LUVOIR. Recently detected by Bonfils et al. (2017) using the radial velocity (RV) measurements, Ross 128 b has a minimum mass of 1.27 M ⊕ and an orbital semi major axis of 0.05 au, which is within the habitable zone of the cool (T eff = 3200 K) M4V dwarf located at 3.38 pc. Ross 128 is representative of potentially habitable planet host stars that may be targeted with future VFN instruments on large-aperture ground-and space-based telescopes. We first determine the required SNR per spectral channel and then calculate the integration time needed to detect the planet in the presence of stellar photon noise as a function of the telescope size, wavelength, and wavefront aberrations.

Required SNR per spectral channel
Detecting molecules in the atmosphere of Ross 128 b with a VFN will require cross-correlating the measured high-resolution spectrum with a template spectrum in order to make efficient use of the relatively few planet photons and the spectral information they carry. A peak in the cross-correlation function (CCF) indicates the presence of a planet with a spectrum similar to the template. In practice, the planet is characterized by modeling the spectra of planets with a variety of possible compositions and finding the template that maximizes the peak in the CCF (Wang et al. 2017;Hoeijmakers et al. 2018).
Using the methods developed in Wang et al. (2017), we determine the relationship between the SNR of the peak in the CCF and the SNR per spectral channel for . Signal-to-noise ratio (SNR) of the peak in the cross-correlation function (CCF) as a function of the SNR per spectral channel from simulated observations in r, J, H, K, and L bands, using low-cloud Earth spectrum and the simulations methods developed in Wang et al. (2017). The required SNR per spectral channel for a CCF SNR of 5 in J band is 0.06, whereas 0.11-0.16 is required in the remaining bands. The actual values are listed in Table 3.
an R=100,000 spectrograph. The spectrum and matching template for an Earth-like planet covered by low clouds orbiting an M dwarf is simulated using an atmospheric chemistry and radiative transfer model (Hu et al. 2012a(Hu et al. ,b, 2013Hu & Seager 2014). The resulting geometric albedo is ∼0.1. Assuming a planet radius of 1.5R ⊕ , the planet-to-star flux ratio in reflected light is ∼5×10 −8 . The assumed stellar spectrum is from the PHOENIX BT-Settl model grid (Baraffe et al. 2015). We calculate the distribution of peak values of the resulting CCFs as a function of the SNR per spectral channel for 100 observations in a set of wavelength ranges that correspond to r, J, H, K, and L bands (see Table 2). In practice, the number of pixels in the spectrograph or atmospheric dispersion will likely limit the maximum spectral bandwidth. Assuming that stellar Table 3. The signal to noise ratio (SNR) per spectral channel and minimum integration time needed to detect Ross 128 b in r, J, H, K, and L bands at a CCF SNR of 5 ignoring detector noise, background noise, and wavefront aberrations. This fundamental limit is set by stellar photon noise due to the diameter of the star (0.5 mas). Whereas J band wavelengths are optimal for observing Ross 128 b with Keck and LUVOIR, the integration time is smaller at longer wavelengths (K and L bands) on TMT.

Goal
Minimum photon noise is the dominant noise source and that the planet spectrum is known, Figure 7 shows the resulting CCF SNR, defined as the ratio between the median of the peak values and the standard deviation of the distribution of peak values. We find that the ratio between the CCF SNR and the SNR per spectral channel is the highest for J band. The required SNR per spectral channel for a CCF SNR of 5 in J band is 0.06, whereas 0.11-0.16 is required in the remaining bands (see Table  3). These values are higher when the template spectrum differs from the true planet spectrum, as discussed in Wang et al. (2017).

Keck telescope
The 10 m Keck telescope is an ideal platform to demonstrate VFN, paving the way for the characterization of Earth-sized planets with future telescopes, such as LUVOIR and TMT. Furthermore, VFN may be implemented as part of the fiber injection unit (FIU) of the Keck Planet Imager and Characterizer (KPIC) with minimal modification to the current instrument design (Mawet et al. 2016;Mawet et al. 2017b).
Ross 128 b has an angular separation of ∼15 mas at maximum elongation, which determines η p as a function of wavelength for the given telescope size. The minimum possible η s is set by the angular diameter of the star: ∼0.5 mas. With the VFN performance parameters listed in Table 2, we computed the minimum integration time for detecting Ross 128 b at a CCF SNR of 5 with the template-matching technique. In cases that require more than ∼6 hr of integration time, the full observing campaign is made up of the summation of a number of six-hour observations taken on multiple nights. The Table 4. Parameters used in integration time calculations. We adopt the detector noise characteristics for a science grade Teledyne H2RG HgCdTe focal plane array cooled to 77 K (Blank et al. 2012) and transmission estimates based on the KPIC instrument. The majority of Ross 128 b parameters are from Bonfils et al. (2017). The planet radius and albedo are currently not well constrained.

Instrument parameters
Telescope transmission, T 0 discrete observations are planned to coincide with the planet's maximum elongation. About that point, assuming Ross 128 b is in an edge-on circular orbit, the angular separation only changes by 0.3% in six hours and therefore has a negligible effect on the integration time estimate. The combined effect of the VFN, the late spectral type of the star, and the SNR gain achieved with templatematching make J band the optimal wavelength range for observing Ross 128 b with Keck (see Table 3). Spectroscopy in J band also provides a means to measure the abundance of H 2 O, O 2 , and CO 2 which are key to assessing habitability (Meadows et al. 2018). The integration times in Table 3 assume the planet spectrum is known and do not include the effect of wavefront error, which is the dominant contributor to η s . Figure 8 shows an estimate of the integration time for achiev- We assume 5 nm rms of coma for the ground-based telescopes (Keck and TMT) and 1 nm rms for space telescopes (LUVOIR). Other relevant assumptions are given in Table 4.
ing a CCF SNR of 5 as function of rms tip-tilt jitter, assuming coma aberrations are controlled to 5 nm rms with Keck AO or an upgraded version thereof. The remaining assumptions are listed in Table 4. Controlling tip-tilt to 10 −2 λ/D rms and coma to 5 nm rms yields an integration time of ∼400 hr. Current state-of-the-art AO systems (e.g. SCExAO; Jovanovic et al. 2015) are capable of controlling tiptilt errors to ∼ 10 −2 λ/D rms and low-order Zernike modes to ∼50 nm rms (Singh et al. 2015). Although the wavefront error requirements shown in Fig. 8 are 10× smaller than currently achieved on SCExAO, improved correction is expected with closed-loop predictive control. Males & Guyon (2018) recently showed that starlight suppression may be improved by a factor of 1400× for high-contrast imaging with a coronagraph and a bright guide star, which corresponds to a reduction in low-order wavefront error by a factor of approximately √ 1400 = 37×. Significant reductions in loworder wavefront error are especially feasible for a VFN instrument since only a single mode (coma) needs to be corrected to higher precision than current AO systems provide.

Thirty Meter Telescope (TMT)
The angular separation of Ross 128 b is approximately the VFN outer working angle with TMT at H band where the throughput begins to drop rapidly (see Table 2). On the other hand, thermal background noise (not included in Table 3) dominates the photon noise budget at L and longer wavelengths. Thus, K band is the optimal wavelength range for observing Ross 128 b with VFN on TMT.
Provided the same AO performance is achieved with TMT as assumed above for Keck (i.e. 10 −2 λ/D rms tip-tilt jitter and 5 nm rms of coma), Fig. 8 shows that the integration time needed to detect Ross 128 b with a CCF SNR of 5 in K band is 100 hr, approximately four times less than with Keck in J band.
K-band observations with TMT will be especially important for detecting CH 4 . Additionally, K-band observations will constrain the abundance of H 2 O and CO 2 , but are not sensitive to O 2 . Therefore, conventional coronagraph observations at J band will complement VFN observations at K band. In fact, we confirmed that conventional coronagraphs provide shorter integration times than VFN in J band and at smaller wavelengths: for instance, using a vortex coronagraph and FIU (Mawet et al. 2017a), due to the improvement in throughput that the coronagraph provides for angular separations > λ/D.

LUVOIR space telescope
The 15 m LUVOIR space telescope may be the most efficient platform for characterizing Earth-sized planets in the habitable zone of M dwarfs with a VFN instrument. Since imaging Earth-like planets around sun-like stars is a premier science goal of LUVOIR, the telescope already has strict stability requirements and extremely precise wavefront control capability (<10 pm rms). The integration time for detecting Ross 128 b would likely reach the fundamental limit set by the finite size of the star.
Similar to the Keck telescope, the optimal wavelength range for observing Ross 128 b with LUVOIR is J band. With the assumptions in Table 4, a CCF SNR of 5 is achieved in only 30 hr of integration. As mentioned above in the context of Keck observations, J-band VFN observations also provide the opportunity to measure the abundance of potential biosignatures H 2 O, O 2 , and CO 2 .
Compared to imaging Earth-like planets in the habitable zone of sun-like stars, where a planet-to-star flux ratio of = 10 −10 is expected, VFN observations of M dwarfs with LUVOIR will not require the time-consuming wavefront control calibrations to dig a dark hole in the speckle pattern at the coronagraph image plane. VFN would thereby be an efficient method for surveying nearby M dwarfs to detect new planets and following up known planets detected through direct imaging at shorter wavelengths, RV measurements, or Gaia astrometry.

Uncertainty in integration time calculations
A major source of uncertainty in the Ross 128 b integration time calculations is that the planet radius, r p , is poorly constrained by its RV detection. Since the integration time scales inversely as the square of the flux ratio (τ ∝ −2 ) and the flux ratio scales as the square of the planet radius ( ∝ r 2 p ), the dependence of integration time on planet radius is a fourth-order power-law relationship: τ ∝ r −4 p . For instance, a planet that is twice the size would take 1/16th of the amount of time to detect. The measured minimum mass of 1.27 M ⊕ implies that Ross 128 b is slightly larger than Earth. We adopted r p = 1.5R ⊕ in the calculations above. However, assuming the orbit is edge-on and assuming the planet has the same density as Earth, the radius of the planet is r p = (1.26) 1/3 R ⊕ = 1.1R ⊕ and the integration time would be 3.4× greater.
Likewise, the planet albedo, α, is also unknown. We assumed a value of α ≈ 0.1 based on a low-cloud atmospheric chemistry and a radiative transfer model (Hu et al. 2012a(Hu et al. ,b, 2013Hu & Seager 2014). However, we find that the albedo may be five times greater for a high-cloud model. Since the integration time scales as τ ∝ α −2 , the integration time could be up 25× shorter with high clouds. On the other hand, the spectral lines differ considerably between low-cloud and high-cloud models, which alters relationship between the CCF SNR and SNR per spectral channel, introducing additional uncertainty.
The assumed orbital inclination affects both the assumed mass and the orbital phase function, φ. However, the true mass could be much greater, in which case, the planet may be larger in size and detected in a much shorter integration time. In addition, the integration time is inversely proportional to the phase function, τ ∝ φ −2 , and therefore the integration time can vary by a factor of ∼10 depending on the planet's orbital inclination and the position along its orbit.
Another source of uncertainty is the performance characteristics of future telescopes and instruments. In the calculations above, we assume a transmission of T = 0.3 (including the telescope and spectrograph) and detector quantum efficiency of q = 0.85. The former is based on end-to-end transmission estimates of the Keck telescope and the KPIC instrument. The latter is representative of a Teledyne H2RG HgCdTe focal plane array (Blank et al. 2012). However, these values can potentially be higher in a dedicated VFN instrument with fewer reflections, higher throughput spectrographs, and improved detectors. The integration time is inversely proportional to each of these quantities.

Detecting new Earth-like planets orbiting M stars
The approach outlined for the Ross 128 b case above may potentially be used to detect new Earth-like planets in the habitable zone of other nearby M stars. We estimated the number of stars around which a CCF SNR of 5 could be achieved for an Earth-sized planet at the Earth isolation distance with the same spectrum, template, and noise properties used above. Using a VFN instrument mode in J band, 10, 15, and 30 m telescopes can respectively search 3, 7, 45 habitable zones for such planets in less than 50 hr. Whereas the targets for a 10-meter telescope are within 3 pc, the habitable zones accessed by a 30-meter are around stars as far as 8 pc.

New scientific opportunities
In addition to enabling the characterization of Earthsized planets in the habitable zone of nearby M dwarfs, a VFN instrument would enable new scientific opportunities for current and future space telescopes.
Upgrading the FIU of KPIC (Mawet et al. 2016;Mawet et al. 2017b) would simply require a new focusing lens. This new mode would allow for the detection of all planet types. Blind surveys of nearby stars may yield new detections of giant planets on close-in orbits and the chemical makeup of their atmospheres. However, a more effective approach is to perform targeted characterization of known planets detected via direct imaging at shorter wavelengths, RV (with e.g. SPIRou; Artigau et al. 2014), and astrometry (with e.g. Gaia; Perryman et al. 2014). The high-resolution spectrum also provides constraints on the orbital radial velocity (Snellen et al. 2014) and, with sufficient SNR, the rotation rate (Bryan et al. 2017). Larger aperture telescopes, such as TMT, will extend the above possibilities to planets with smaller values and angular separations.

Potential design improvements
Although the F # of the system has been chosen to maximize the coupling of planet light into the SMF, it is possible that varying the F # may lead to lower integration times under a given set of wavefront error assumptions (Mennesson et al. 2002). In addition, beam shaping techniques, such as phase induced amplitude apodization, may be a pathway to improve throughput (Jovanovic et al. 2017).
Efficient VFN observations require very tight constraints on tip-tilt and coma wavefront errors. Fortunately, future VFN instruments can take advantage of recent developments in low-order wavefront sensing and control, including reflective Lyot stop (Singh et al. 2015) and holographic wavefront sensors designed to sense the exp(±iθ) modes upstream of the vortex phase mask (Wilby et al. 2017). All of the above will benefit from closed-loop predictive control under development on state-of-the-art AO systems (Males & Guyon 2018).
With low-noise, high-speed infrared detectors (e.g. Greffe et al. 2016), it is possible to obtain spectra at high enough frame rates (>kHz) to prevent averaging over the full distribution of tip-tilt and coma errors. Excluding frames where the errors are greater than a given threshold may reduce the effective η s and thereby the integration time needed for detection. A similar approach was presented in Hanot et al. (2011).

Limitations
VFN is fundamentally limited in terms of outer working angle. Making λF # smaller with respect to the fiber mode incurs significant throughput losses. It is more efficient to characterize planets outside of λ/D with point spectroscopy downstream of a conventional high-contrast imager.
Another major limitation for VFN is that the integration time needed to characterize planets at < 10 −8 becomes longer than a feasible observing program. For example, the signal from Earth-like planets in the habitable zone of sun-like stars with ≈ 10 −10 will be overwhelmed by stellar photon noise owing to the finite size of the star (∼1 mas) and would require >1000× longer integration times than Ross 128 b. That science case calls for a larger inner working angle coronagraph, such as the vortex coronagraphs proposed for the Habitable Exoplanet Imaging Mission (HabEx; Mennesson et al. 2016;Ruane et al. 2018).

CONCLUSION
The vortex fiber-nulling (VFN) method uses the combination of a vortex phase mask and single-mode fiber to reject starlight, while feeding light from planets at angular separations of λ/D to a spectrograph. Combined with the template-matching technique at high spectral resolution, VFN provides a pathway to characterize Earth-sized planets in the habitable zone of nearby M dwarfs, such as Ross 128 b, in ∼400, ∼100, and ∼30 hr with Keck, TMT, and LUVOIR, respectively. The integration time is strongly dependent on the planet properties as well as tip-tilt and coma aberrations but is otherwise insensitive to wavefront errors. VFN requires wavefront residuals of ∼ 10 −2 λ/D rms of tip-tilt and on the order of a few nm rms of coma. The finite size of the star dominates the stellar photon noise budget in the case for ultra-stable space telescopes designed for exoplanet imaging, such as LUVOIR. VFN enables efficient, targeted characterization of known planets orbiting nearby stars. A laboratory demonstration of the VFN technique is underway at Caltech's Exoplanet Technology (ET) Laboratory (Delorme et al. 2018).
G. Ruane is supported by an NSF Astronomy and Astrophysics Postdoctoral Fellowship under award AST-1602444.