Improved Correlated Multiple Sampling by Using Interleaved Pixel Source Follower for High-Resolution and High-Framerate CMOS Image Sensor

This article describes an improvement in the noise reduction performance of a column correlated multiple sampling (CMS) readout circuit using interleaved pixel source follower for high-resolution and high-framerate CMOS image sensors (CISs). In this architecture, the time-interleaved operation of the two pixel source followers reduces the restrictions imposed by the settling time of the pixel source followers and extends the time for multiple sampling. The noise analysis indicates that this method has an advantage of enhanced noise reduction not only for thermal noise but also for 1/ ${f}$ noise when a high-speed readout operation is required. The measurement of the noise performance of the 8K image sensor using the CMS with the interleaved pixel source follower method exhibits a low input-referred noise of 3.2 e− at 8K 120 frames per second, while 4.6 e− with the conventional single-source follower readout method. The measurement results match reasonably well with the analysis presented in this article, demonstrating the effectiveness of the interleaved pixel source follower method for high-resolution and high-framerate CISs.


Improved Correlated Multiple Sampling by Using
specify spatial resolutions of up to 8K and frame frequencies of up to 120 frames per second (fps). One of the most challenging tasks for such high-resolution and high-framerate image sensors is to achieve high-speed readout and low-noise performance. For example, the single-row time of an 8K 120 fps progressive-scan image sensor, which corresponds to the readout time for a single row in a column-parallel readout circuit architecture, is 1.85 μs (including vertical blanking interval), which is much shorter than that of a high-definition television (HDTV) 1080i (29.6 μs) [3]. In addition, a large number of pixels require a small pixel pitch, which leads to degradation of the signal-to-noise ratio.
The correlated multiple sampling (CMS) technique has been studied as a promising noise reduction technique for achieving low-noise performance characteristics in CMOS image sensors (CISs) [4]- [6]. CMS is advantageous owing to its ability to efficiently reduce both thermal and 1/ f noise of the pixel source follower amplifiers, which are known to be one of the major noise components in well-designed lownoise CISs [7]. In the past few years, the CMS column-parallel analog-to-digital converters (ADCs) have been implemented with the CMS technique in various ways. These include a digital implementation with multiple A/D conversions based on a single slope (SS) [8], [9] and successive approximation register (SAR) [10] ADC and an analog implementation with a passive switched capacitor (SC) [11], [12] and an SC integrator circuit [13], [14].
However, achieving both high-speed readout and low noise, which are required in high-resolution and high-framerate CISs, is difficult for the CMS readout circuit. Indeed, they have not been used in CISs that meet both the pixel count of >8 Mpixel and framerate of >60 fps. One of the reasons for this is the restriction imposed by the settling time of the pixel source follower. Since multiple sampling must be performed after the pixel source follower output has settled, a part of the readout time is devoted to the settling of the pixel source follower. In particular, for high-resolution and high-framerate CISs, the readout time is limited to a short period and the settling time of the pixel source follower tends to be long because of the parasitic capacitance due to a large number of This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ vertical pixels. These factors make it difficult to secure the time required for multiple sampling.
To overcome this difficulty, we previously implemented a circuit topology with interleaved pixel source followers in a column CMS readout circuit [15]. In this architecture, the twopixel source followers work in parallel at different phases: the output of one is multiple sampled while settling the output of the other. This time-interleaved operation reduces the restriction imposed by the settling time of the pixel source followers and extends the time for multiple sampling. We applied this method to a column-parallel readout circuit in an 8K image sensor and achieved random noise of 3.2 e − at a readout time of 0.93 μs [8K 120 fps operation with digital correlated double sampling (CDS)] [15].
However, the precise contribution of the interleaved pixel source follower method has not been evaluated. This method enhances the reduction effect, especially for thermal noise, as thermal noise is known to be reduced by a factor equal to the square root of the sampling number [16]. In contrast, the reduction effect for 1/ f noise could be degraded even when the sampling number is increased. This is because increasing the sampling number certainly reduces the 1/ f noise, but simultaneously the parallel operation of pixel source followers increases the interval of the reset and signal sampling, which is known to limit the reduction effect for 1/ f noise [5]. These conflicting effects are affected by the readout speed because increasing the sampling number and the interval of the reset and signal sampling are strongly dependent on the readout speed. Therefore, to clarify the precise contribution of the interleaved pixel source follower method, we need to discuss further both the thermal and 1/ f noise reduction effects considering its dependence on the readout speed.
In this article, the impact of the interleaved pixel source follower method on both thermal and 1/ f noise is theoretically analyzed. Moreover, its dependence on the operation speed is discussed. To verify the theoretical analysis, we measured and compared the theoretical calculations with the noise performance of an 8K image sensor implemented with the interleaved pixel source follower method. The contribution of the interleaved pixel source follower method for the noise performance in high-resolution and high-framerate CISs is demonstrated.

II. THEORETICAL NOISE ANALYSIS
In this section, the noise reduction effect of the interleaved pixel source follower method for both the thermal and 1/ f noise is theoretically analyzed and compared with that of the conventional method regarding their dependencies on the operation speed. In the following discussion, a column CMS readout circuit is implemented with an SC integrator, column ADC, and digital CDS circuit as a typical example of the architecture.

A. Architecture
The schematic and timing diagrams of a conventional CMS readout circuit are shown in Fig. 1. The reset and signal levels of the source follower output (V R and V S , respectively) are sampled and integrated M times by the SC integrator. The column ADC converts the output voltages of the SC integrator into the digital code. The digital CDS circuit takes the difference between the digital code of V R and V S . T H , T 0 , and T sett are the readout time for a single row, the sampling period of the multiple sampling, and the settling time of the pixel source follower, respectively. For simplicity, T sett and T H /2 are assumed to be integral multiples of T 0 . Subsequently, considering the time constraint, the sampling number M and the interval of the reset and signal multiple sampling T g can be expressed as As shown in (1), a part of the readout time is devoted to the settling of the pixel source follower. Therefore, shorter T H and longer T sett , which are required for higher resolution and framerate CISs, result in a smaller M. This relationship makes it difficult to implement the CMS technique for high-resolution and high-framerate CISs. Fig. 2 shows a schematic and a timing diagram of a readout circuit using the interleaved pixel source follower method. Each pixel column has two source followers (SFA and SFB). Column vertical pixels are divided line by line into pixels A and B, and they are alternately connected to the two source followers. The two source followers work in parallel, and the phase of their operation differs by T H /2. The outputs of the source followers are connected to the SC integrator via selector switches ( SFA and SFB ). Subsequently, the SC integrator receives the reset levels of pixels A and B (V RA and V RB ), followed by their signal levels (V SA and V SB ). The digital CDS circuit takes the difference between the digital code of V SA and V RA and between that of V SB and V RB using a set of two registers. In this architecture, the input voltages of the SC integrator have already been settled before connecting to the SC integrator, which reduces the waiting time for settling. We call this topology the "interleaved pixel source follower method." T t is the minimum time interval between the last Mth sampling point and the first sampling point of the next multiple sampling of the SC integrator. This is the time that is required to operate the following operations: integrating the Mth sampled signal, column ADC sampling, resetting the SC integrator output, and charging the input capacitor for the first sampling of the next multiple sampling. Notably, T t < T sett /2 is required to achieve M = 1.
When T g = T sett (4) and Note that T t is assumed to be an integral multiple of T 0 for simplicity.

B. Thermal Noise Reduction Effect
In the CMS readout circuit, the thermal noise is reduced by a factor equal to the square root of the sampling number owing to the averaging effect of multiple sampling [16]. Therefore, the enhancement of the noise reduction effect caused by the interleaved source follower can be calculated by the increase in M from the conventional method. In the conventional method, because a part of the readout time is devoted to the settling of the pixel source follower, M decreases as T sett increases, as shown in (1). In the interleaved pixel source follower method, T t has the same relationship with M as T sett has in the conventional method when T H /2 ≥ T sett − T t , as shown in (5). In high-resolution CISs, T t < T sett is supposed to be because of the large parasitic capacitance as a large number of vertical pixels strongly limits the bandwidth of the pixel source follower, which results in a large T sett . However, T t can be optimized by the SC integrator design regardless of the number of vertical pixels. The amount of increase in M from the conventional method to the interleaved pixel source follower method, that is, M, is obtained using (1) and (5) and is expressed as Here, T H /2 ≥ T sett −T t is assumed. Fig. 3 compares the thermal noise reduction effect of the conventional and interleaved pixel source follower methods. The term 1/ √ M is plotted as a function of T H /T 0 to discuss their dependence on the readout time. Here, T 0 , T sett , and T t are treated as constants, and T H is treated as a variable. T sett = 7T 0 and T t = 3T 0 are assumed as examples, which correspond to the actual value designed for the 8K image sensor described in Section III-A. T sett , T t , and T 0 strongly depend on the pixel rate and various design constraints, such as power consumption and circuit area. M is calculated as (4) from (7). The markers in Fig. 3

C. 1/f Noise Reduction Effect
T g is constant (=T sett ) regardless of T H in the conventional method, while it increases with T H in the interleaved pixel source follower method. This can lead to the degradation of the noise reduction effect for the 1/ f noise component in the interleaved pixel source follower method. The noise reduction effect of the CMS for 1/ f noise can be expressed by the noise reduction factor F CMS defined by [5] with the definition of x c = ω c T 0 and x = ωT 0 , where ω c is the cutoff angular frequency determining the bandwidth of the noise components, and M g is an integer defined by M g = T g /T 0 . F CMS can be approximated by a noise reduction factor of the differential averager F DA expressed by [17] where R G is the ratio of M g to M (R G = M g /M). This approximation is useful for calculating F CMS as a function of only R G without numerical calculation of (8). For a large M, F CMS can be exactly approximated by F DA . R G of the conventional and interleaved pixel source follower methods (R G,S and R G,D , respectively) can be obtained from (1), (2), (5), and (6) as expressed by Here, T H /2 ≥ T sett − T t is assumed. At the point of T H satisfying R G,S = R G,D , F CMS of the two methods can be approximated as roughly equal using (9). For T H /T 0 1 (M 1), it is supposed that R G,S ∼ = 0 and R G,D ∼ = 1;  (8), is plotted as a function of T H /T 0 . T sett = 7T 0 , T t = 3T 0 (the same as in Fig. 3), and x c = 16 are assumed as an example. T H that meets R G,S = R G,D (denoted as T HX ) is plotted as the vertical line of T HX /T 0 ( ∼ =27). The point of intersection of the two lines approaches the line of T HX /T 0 . When T H /T 0 < T HX /T 0 , a higher noise reduction effect can be obtained for the interleaved pixel source follower method. In contrast, when T H /T 0 > T HX /T 0 , the interleaved pixel source follower method shows an inferior reduction effect for the 1/ f noise compared with the conventional method. These are caused by the improvement effect by M that becomes smaller while the degradation caused by T g increases with increasing T H /T 0 . For T H /T 0 1, F CMS of the conventional and interleaved pixel source follower methods saturates to 4ln(2) ∼ = 2.77 and 9ln(3) − 8ln(2) ∼ = 4.34, respectively, which are equal to the saturation values approximated above using (9).

D. Applicability
An overall noise reduction effect can be obtained by applying the reduction factors 1/ √ M and F CMS for thermal and 1/ f noise components, respectively, and summing them. For low-speed CISs, where the thermal noise can be effectively suppressed by an efficient sampling number of the CMS or the bandwidth limitation effect of the low-speed readout circuit, the 1/ f noise could become a major noise component [18]. Under this condition, the interleaved pixel source follower method does not work effectively because the enhancement, owing to M, is small, and the degradation in 1/ f noise reduction that is caused by the increase in T g becomes large. In contrast, for high-speed CISs, only a few sampling numbers can be obtained, and high bandwidth is required for the readout circuit, which leads to large thermal noise components. Under this condition, the interleaved pixel source follower method is effective because the thermal noise reduction effects are effectively enhanced by M. Especially for high-resolution CIS, as shown in (7), M tends to be large because of the large difference between T sett and T t , which results in a large enhancement of the noise reduction effect. In addition, from the discussion on the 1/ f noise reduction effect, if T H is shorter than T HX , the interleaved pixel source follower method enhances the noise reduction effect even for 1/ f noise components. Therefore, M and T HX provide clear indications of the applicability. We conclude that the interleaved pixel source follower method effectively reduces noise at high-resolution and high-framerate CISs.

A. Implementation of 8K Image Sensor
The schematic and timing diagrams of an 8K image sensor readout circuit with an implemented interleaved pixel source follower method are shown in Figs. 5 and 6, respectively. The image sensor has a pixel array with 2.1-μm pixel pitch (effective area: 4.320 pixels × 7.680 pixels). The vertically aligned two-shared pixels are connected alternately to the two source followers. The outputs of the source followers are connected to an analog column CDS circuit via two switches, which is for level adjustment and amplification of the pixel source follower output. Subsequently, the CDS circuit receives the reset levels of pixels 1 and 3 (V R1 and V R3 ), followed by their signal levels (V S1 and V S3 ). Once V R1 is received, capacitor C f is reset by R , and the output voltage of the CDS (V CDS ) is set to a common voltage V com . Thus, the output where G A is the gain of the column CDS (G A = C i/ C f = 2). These voltages are sampled and integrated M times and converted into digital code by the three-stage pipelined ADC, consisting of folding integration (FI) and cyclic, SAR ADC. The FI ADC has an SC integrator that executes multiple sampling. The digital CDS circuit takes the difference between the first and third outputs and between the second and fourth outputs using two sets of registers. Then, the final CMS output codes of G A (V S1 -V R1 ) and G A (V S3 -V R3 ) are obtained. In this implementation, a margin, corresponding to the maximum difference between V R1 and V R3 , is needed for the lower limit of the column ADC input voltage range. This is required because if V R3 -V R1 is negative, then V com + G A (V R3 -V R1 ) is lower than the reference level of the CDS output (V com ). Output voltage clipping circuits are used in the pixel source followers to avoid abnormal voltage fluctuations in V R1 and V R3 . The difference between the two pixel source followers' gains or linearity error of the column ADC can cause horizontal striped artifacts, which can be reduced by applying signal corrections in the digital domain. Fig. 7 shows (a) circuit diagram and (b) timing diagram of the FI ADC. The FI ADC consists of an SC integrator, comparator, counter, and negative feedback path with a 1-bit digital to analog converter (DAC). This ADC works as a first-order incremental delta-sigma modulator. The counter output is treated as higher bits, and the residue voltage of the SC integrator is converted into lower bits by the following ADC. This type of A/D conversion is also known as an extended counting ADC [19], [20]. Duplicated sampling capacitors (C 1A and C 1B ) are implemented to obtain highspeed operation. In the first sampling phase, C 1A and C 2 are charged by the input voltage, and the voltage across C 2 and V RC is compared by the comparator. The sampled charge of C 1A is transferred to C 2 so that the integration gain is G I + 1, where G I = C 1A /C 2 = C 1B /C 2 = 1/2. In the second sampling phase, the charge sampled in C 1A is transferred to C 2 , and a reference voltage (V RL or V RH ) for the 1-bit DAC is subtracted. Moreover, C 1B simultaneously samples the input voltage. In the following phases, the ADC works similarly as in the second sampling phase. However, the roles of C 1A and C 1B are switched. The input voltage is amplified by the gain of G I M + 1, and the output voltage is maintained to a limited range from V RL to V RH .

B. Noise Component Analysis in the Readout Circuit
The noise components of the 8K image sensor are clarified in this section. For this, the noise characteristics of the signal readout circuit from the pixel-to-column ADC were analyzed based on the model presented in [5] with some modifications. The noise power referred to at the CMS output can be expressed as the sum of the noise components described as follows: P n,rst is the reset noise component of the integrator; P nT,smpl and P nF,smpl are the thermal and 1/ f noise components in the sampling phase, respectively; P nT,trns and P nF,trns are the thermal and 1/ f noise components in the signal charge transfer phase, respectively; and P nT,ADC is the thermal noise component of the column ADC sampling. P n,rst , P nT,trns , P nF,trns , and P nT,ADC can be calculated using the same model presented in [5]. In this analysis, P nF,trns and 1/ f noise due to the amplifier of the FI ADC are ignored because the amplifier uses relatively large transistors with low 1/ f noise. P nT,smpl and P nF,smpl must be modified to include the effect of the first sampling and the noise component of the column CDS and column bias circuit. P nT,smpl is modified as P * nT,smpl = P nT,SF + P nT,Bias + P nT,CDS where P nT,SF, P nT,Bias , and P nT,CDS are the noise components generated by the source follower, column bias circuit, and column CDS, respectively. The noise bandwidth limitation of the source follower and column bias circuit are assumed independent of the FI ADC sampling capacitor because the cutoff frequencies of the source follower ω cSF and the column bias circuit ω cB are lower than that of the CDS circuit. Thus, P nT,SF , and P nT,Bias can be calculated by where G nSF is the noise gain factor of the source follower [5], ξ SF and ξ B are the excess thermal noise factor of the source follower and the column bias circuit, respectively; g mSF , g mCS , and g mBias are the transconductance values of transistors M1, M6, and M7, respectively, as shown in Fig. 5, and H * CMS (ω) is the CMS transfer function. From the sampling timing and integration gain of the FI ADC, H * CMS is obtained with the z transform and is expressed in the z domain as where M g is an integer defined by M g = T g /T 0 . T g and T 0 are the intervals of the reset and signal multiple sampling and the sampling period, respectively. With z = exp( j ωT 0 ), the noise power transfer function for the CMS H * CMS (ω) is given by Similar calculations cannot be applied to P nT,CDS because the CDS cutoff frequency is affected by the sampling capacitance of the FI ADC. However, each sampled noise of the first sampling (P n,CDS1 ) and second to Mth samplings (P n,CDS2 ) can be calculated separately, and P nT,CDS can be obtained by summing them where and where ξ CA , β A , and C L are the excess thermal noise factor, feedback factor, and load capacitance of the amplifier, respectively, which are used for the CDS circuit in the Mth sampling phase. P nF,smpl is modified as follows: Timing diagram of the readout circuit used for emulating the conventional readout timing. The same operation is conducted for pixels 2 and 4.
where K fSF is the flicker noise coefficient, and ς SF is the flicker noise factor of the source follower. The total noise power referred to at the output of the readout circuit P nC,total is given by P n,total = P n,rst + P * nT,smpl + P * nF,smpl + P nT,trns + P nT,ADC .
The gain from the charge generated in the photodiode to the output is given by G cSF G A (G I M + 1), where G cSF is the conversion gain of the pixel source follower. The input referred noise is given by

C. Experimental Setup
The sampling timings used for the measurement are shown in Table I. The readout time for a single row T H of 1.85, 3.70, and 7.41 μs, corresponding to framerates of 120, 60, and 30, respectively, is used for the measurement. The conventional readout operation was emulated using the same readout circuit to compare with the conventional method. Fig. 8 shows the readout timing used for emulating the conventional readout operation. The two pixel source followers were operated one by one. The select switches ( SFA and SFB ) select the SFA, and the reset and signal levels of pixel 1 were readout in the period of T H . Subsequently, the select switches select SFB, and the reset and signal levels of pixel 3 were readout in the next period of T H . The column CDS circuit receives the reset and signal level of the pixels subsequently, and the digital CDS circuit takes the difference between the two consecutive codes. Note that the same readout operation can be performed by a single-pixel source follower and a single register in the digital-CDS circuit used for the conventional readout circuit. T 0 , T t , and T sett were 0.11, 0.81, and 0.27 μs, respectively. These parameters were determined by the bandwidth of the pixel source follower and SC integrator in the FI ADC. M obtained in the interleaved pixel source follower method is M (=4) larger than that of the conventional method for the same T H . T g is equal to T sett (=0.81 μs) regardless of T H in the conventional readout method; however, it increases in proportion to T H in the interleaved pixel source follower method. To measure dark noise for the 8K image sensor, the extracted 100 × 100 pixels were used to analyze the random noise; the root-mean-square noise values were calculated for each pixel, and the median value of the pixels was treated as random noise. Fig. 9 shows the calculated noise components as a function of M, where N n,rst , N * nT,smpl , N * nF,smpl , N nT,trns , and N nT,ADC are the input-referred noise components of P n,rst , P * nT,smpl , P * nF,smpl , P nT,trns , and P nT,ADC , respectively. Moreover, the sum of them is plotted as N n,total . Because N n,rst , N * nT,smpl , N nT,trns , and N nT,ADC showed the same behavior for both the conventional and interleaved pixel source follower methods, they are plotted using similar lines for both the methods. This is because these components are dependent on M but are independent of T g [5]. In contrast, N * nF,smpl that originates from the 1/ f noise of the source follower amplifier shows a different tendency when comparing the two methods. This is because the 1/ f noise reduction effect is affected by T g , as discussed in Section II. When the sampling number is small, N * nT,smpl is a dominant component; however, it is suppressed with an increase in M. For large M, N * nF,smpl becomes dominant, which is because the noise reduction effect for 1/ f noise is saturated for a large M. The saturation value of the interleaved pixel source follower method is higher than that of the conventional method, which corresponds to the degradation of the 1/ f noise reduction effect caused by the interleaved pixel source follower method, as presented in Section II. The noise components N n,rst , N nT,trns , N nT,ADC , and N nT,smpl are smaller than N * nT,smpl and effectively suppressed with an increase in M. From these results, N * nT,smpl and N * nF,smpl can be regarded as the main noise components that can explain the behavior of the overall noise performance of the image sensor. Fig. 10 compares the measured input-referred noise of the conventional and interleaved pixel source follower methods as a function of T H . The calculated noise of N n,total is plotted to compare with the calculation results. N * nT,smpl and N * nF,smpl are plotted as the main noise components of N n,total . At T H = 1.85 μs, corresponding to 8K 120 fps operation, a low input-referred noise of 3.2 e − with M = 6 is obtained for the interleaved pixel source follower method, while that for the conventional method is 4.6 e − with M = 2. The difference in the input referred-noise between the two methods decreases with an increase in T H . At T H = 7.41 μs, corresponding to 8K 30 fps operation, 1.5 e − with M = 30 and 1.6 e − with M = 26 are obtained for the interleaved pixel source follower method and the conventional method, respectively. These results can be explained by the analysis presented in Section II as follows. For high-speed readout operation (at T H = 1.85 μs), because T H is shorter than T HX , M (=4) strongly enhances the noise reduction performance for both the thermal and 1/ f noise, which results in the low input-referred noise of the interleaved pixel source follower method. For a relatively low-speed readout operation (at T H = 7.41 μs), the improvement effect by M becomes small, and the degradation of 1/ f noise reduction effect caused by T g becomes larger. Therefore, the difference in the input-referred noise between the two methods becomes small. N * nT,smpl , N * nF,smpl , and thermal and 1/ f noise components in the sampling phase, respectively, show the behavior mentioned above, and N n,total , whose main component can be regarded as N * nT,smpl and N * nF,smpl , agrees reasonably well with the measured results. Thus, these results clearly demonstrate the contribution of the interleaved pixel source follower method for the improvement of noise performance in the high-resolution and high-framerate CIS.

IV. CONCLUSION
This article describes the noise reduction effect of a column CMS readout circuit with interleaved two pixel source followers and discusses its effectiveness for high-resolution and high-framerate CISs. The noise reduction effect analysis indicated that the increase in the sampling number, owing to the interleaved pixel source followers, tends to be large in the high-resolution CISs, resulting in a large enhancement in noise reduction. Furthermore, the interleaved pixel source follower method has the advantage of enhanced noise reduction performance not only for thermal noise but also for 1/ f noise when a high-speed readout operation is required. The measurement of the noise performance for the 8K image sensor implemented with the interleaved pixel source follower method showed the low input-referred noise of 3.2 e − at 8K 120 fps operation while 4.6 e − for the conventional readout method. Furthermore, their dependence on the readout speed and the difference between the two methods agreed reasonably well with the analysis presented in this article. These results demonstrated the effectiveness of the interleaved pixel source follower method and clarified its effect on the noise performance at high-resolution and high-framerate CIS.