RGB visible light communication using mobile-phone camera and multi-input multi-output

Red, green, blue (RGB) light-emitting-diodes (LEDs) are used to increase the visible light communication (VLC) transmission capacity via wavelength-division-multiplexing (WDM), and the color image sensor in mobile phone is used to separate different color signals via a color filter array. However, due to the wide optical bandwidths of the color filters, there is a high spectral overlap among different channels, and a high interchannel interference (ICI) happens. Here, we propose and demonstrate an RGB VLC transmission using CMOS image sensor with multi-input multioutput (MIMO) technique to mitigate the ICI and retrieve the three independent color channels in the rolling shutter pattern. Data pattern extinction-ratio (ER) enhancement and thresholding are deployed. ©2016 Optical Society of America OCIS codes: (230.3670) Light-emitting diodes; (060.4510) Optical communications; (060.4080) Modulation. References and links 1. S. Wu, H. Wang, and C. H. Youn, “Visible light communications for 5G wireless networking systems: from fixed to mobile communications,” IEEE Netw. 28(6), 41–45 (2014). 2. H. Haas and C. Chen, “What is LiFi?” in Proc. ECOC (2015), paper 0871. 3. C. W. Chow, C. H. Yeh, Y. Liu, and Y. F. Liu, “Digital signal processing for light emitting diode based visible light communication,” IEEE Photonics Soc. Newslett. 26, 9–13 (2012). 4. B. Janjua, H. M. Oubei, J. R. D. Retamal, T. K. Ng, C.-T. Tsai, H.-Y. Wang, Y.-C. Chi, H.-C. Kuo, G.-R. Lin, J.-H. He, and B. S. Ooi, “Going beyond 4 Gbps data rate by employing RGB laser diodes for visible light communication,” Opt. Express 23(14), 18746–18753 (2015). 5. Y. C. Chi, D. H. Hsieh, C. T. Tsai, H. Y. Chen, H. C. Kuo, and G. R. Lin, “450-nm GaN laser diode enables high-speed visible light communication with 9-Gbps QAM-OFDM,” Opt. Express 23(10), 13051–13059 (2015). 6. W. Y. Lin, C. Y. Chen, H. H. Lu, C. H. Chang, Y. P. Lin, H. C. Lin, and H. W. Wu, “10m/500 Mbps WDM visible light communication systems,” Opt. Express 20(9), 9919–9924 (2012). 7. Z. Wang, C. Yu, W. D. Zhong, J. Chen, and W. Chen, “Performance of a novel LED lamp arrangement to reduce SNR fluctuation for multi-user visible light communication systems,” Opt. Express 20(4), 4564–4573 (2012). 8. I. Takai, S. Ito, K. Yasutomi, K. Kagawa, M. Andoh, and S. Kawahito, “LED and CMOS image sensor based optical wireless communication system for automotive applications,” IEEE Photonics J. 5(5), 6801418 (2013). 9. P. Luo, M. Zhang, Z. Ghassemlooy, H. L. Minh, H. M. Tsai, X. Tang, L. C. Png, and D. Han, “Experimental demonstration of RGB LED-based optical camera communications,” IEEE Photonics J. 7, 7904242 (2015). 10. C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” in Proc. OWC (2012), pp. 1244–1248. 11. C. W. Chow, C. Y. Chen, and S. H. Chen, “Visible light communication using mobile-phone camera with data rate higher than frame rate,” Opt. Express 23(20), 26080–26085 (2015). 12. C. W. Chow, C. Y. Chen, and S. H. Chen, “Enhancement of signal performance in LED visible light communications using mobile phone camera,” IEEE Photonics J. 7(5), 7903607 (2015). 13. Y. Liu, H. Y. Chen, K. Liang, C. W. Hsu, C. W. Chow, and C. H. Yeh, “Visible light communication using receivers of camera image sensor and solar cell,” IEEE Photonics J. 8(1), 7800107 (2016). 14. J. Y. Sung, C. W. Chow, and C. H. Yeh, “Dimming-discrete-multi-tone (DMT) for simultaneous color control and high speed visible light communication,” Opt. Express 22(7), 7538–7543 (2014). 15. S. Rajagopal, R. Roberts, and S. K. Lim, “IEEE 802.15.7 visible light communication: modulation schemes and dimming support,” IEEE Commun. Mag. 50(3), 72–82 (2012). 16. H. H. Lu, Y. P. Lin, P. Y. Wu, C. Y. Chen, M. C. Chen, and T. W. Jhang, “A multiple-input-multiple-output visible light communication system based on VCSELs and spatial light modulators,” Opt. Express 22(3), 3468– 3474 (2014). #256808 Received 7 Jan 2016; revised 22 Feb 2016; accepted 23 Feb 2016; published 20 Apr 2016 © 2016 OSA 2 May 2016 | Vol. 24, No. 9 | DOI:10.1364/OE.24.009383 | OPTICS EXPRESS 9383 17. C. H. Chang, C. Y. Li, H. H. Lu, C. Y. Lin, J. H. Chen, Z. W. Wan, and C. J. Cheng, “A 100-Gb/s multiple-input multiple-output visible laser light communication system,” J. Lightwave Technol. 32(24), 4723–4729 (2014). 18. M. El-Desouki, M. J. Deen, Q. Fang, L. Liu, F. Tse, and D. Armstrong, “CMOS image sensors for high speed applications,” Sensors 9(1), 430–444 (2009).


Introduction
As there is a shortage of radio-frequency (RF) spectrum for the traditional wireless communications, it is believed that wireless communication using visible light, known as visible light communication (VLC) can become one of the promising candidates for the 5th generation (5G) wireless communication [1] as well as for the internet of things (IOT) applications [2].VLC [3][4][5][6][7] can be integrated with the light-emitting-diode (LED) illuminance systems; hence it can be a value-added service for the lighting industries.VLC is very directional; hence little interference occurs for the neighboring channels.This is an important feature to increase the wireless communication capacity needed for the 5G.For the last decade, the complementary-metal-oxide-semiconductor (CMOS) cameras have been widely used in mobile-phone and vehicles.It is interesting to use these cameras as the receivers (Rxs) for VLC since the VLC deployment cost can be reduced.Although the data rate is limited by the CMOS cameras, it may be enough for sending secure ID, authorization information, or positioning data in indoor navigation system.Ref [8].demonstrated using tailor-made car camera with specific high-speed photo-diode (PD) pixels for VLC and lowspeed pixels for imaging.Ref [9].demonstrated using red-green-blue (RGB) camera for 60 m long VLC transmission, suitable for car-to-car communications.However its data rate is limited to 3 x 50 bit/s (using RGB).Previously, it was demonstrated that the rolling shutter effect of the CMOS image sensor Rx can increase the data rate by using the row-by-row pixel activation [10,11].By demodulating this rolling shutter pattern (bright and dark fringes), ~1 kbit/s data rate can be achieved.Ref [11].analyzed several properties of the rolling shutter effect, and refs [12,13].demonstrated the "blooming" mitigation, extinction-ratio (ER) enhancement and thresholding to improve the transmission performance.
Recently, red, green, blue (RGB) LEDs [14] have been widely used in many places; and the color-tuning ability can provide fancy lighting conditions for different applications.Besides, color-shift-keying (CSK) using RGB has been standardized to provide additional modulation dimensions to enhance the transmission data rate [15].In this work, RGB LEDs are used to increase the VLC transmission capacity by using wavelength division multiplexing (WDM); and the color image sensor can separate different color signals via a color filter array on top of the color-insensitive sensing element.However, due to the wide optical bandwidths of the color filters, there is a high spectral overlap among different channels, and a high WDM inter-channel interference (ICI) happens.Here, we propose and demonstrate for the first time up to the authors' knowledge a RGB VLC transmission using CMOS image sensor with multi-input multi-output (MIMO) technique to mitigate the ICI and retrieve the three independent color channels in the rolling shutter pattern.A second order polynomial fitting is applied to enhance the data pattern ER, and then another second order polynomial fitting is applied for thresholding.Experimental bit-error-rate (BER) performance is improved when MIMO is used, satisfying the forward-error-correction (FEC) limits.

Proposed architecture
As mentioned above, the color camera can separate the RGB signals according to their wavelengths; hence the VLC transmission capacity can be increased.As the CMOS sensing elements are color-insensitive, color filter array on top of the sensing element is used to separate the R, G and B color signals.However, as the optical filter bandwidths are wide, and may not exactly wavelength match with the wavelengths of the RGB LEDs.This will create color misjudge by the image sensor and result in ICI. Figure 1 shows the proposed architecture of the RGB VLC transmission using MIMO to demodulate the rolling shutter pattern.The RGB LED transmitter (Tx) used is from Lumileds Luxeon Z.The color chips used are "deep red", "green" and "royal blue"; and the typical peak wavelengths are 660 nm, 530 nm and 450 nm respectively; with typical spectral half-widths (3-dB bandwidth of optical spectrum) of 20 nm, 100 nm and 20 nm respectively.The color Rx used is an iPhone5 mobile phone with image sensor resolution of 1080 x 1920 pixels.The typical peak color filter wavelengths of the R, G, and B in iPhone5 are 600 nm, 530 nm and 440 nm respectively, with spectral half-widths of 70 nm, 80 nm, and 20 nm respectively.The optical spectra sketches of the LEDs and Rx are shown in Fig. 1(a) and 1(b) respectively.We can observe that there is a high wavelength mismatch between the LEDs and the Rx; and there is a high spectral overlap among different channels in the sensor filter spectra.If there is no spectral overlap of the optical filter bandwidths in the image sensor, and the R, G, B signal line-widths are narrow enough and can match with the optical filter wavelengths; the signal performance will improve and MIMO can be unnecessary.To mitigate the ICI of the received rolling shutter pattern, MIMO technique [16,17] is applied.The optical power received at the Rxs, transmitted by the LED Txs, the channel gain, and the noise can be represented by Y, X, H and N respectively as shown in Eq. (1).
The channel gain matrix H can be represented by Eq. ( 2), where, for example, h rg is the channel gain from the red LED Tx and received by the green Rx.
By neglecting the noise, the channel gain matrix H can be obtained by sending the known training signals from the RGB LED Txs as shown in Eq. ( 3). 1 Finally, the estimated transmitted signal X' can be retrieved from the received signal Y and the obtained channel gain matrix H, as shown in Eq. ( 4).
#256808  When using mobile phone CMOS image sensor as Rx, one difficulty is the frame-to-frame processing time gap during recording.As the RGB LEDs are continuously sending data, there will be a data loss during this frame-to-frame time gap.The iPhone5 has frame rate of 30 fps and processing time gap of ~38% of the frame duration.The frame-to-frame processing time gap can be reduced by better design and architecture of the CMOS readout circuits [18].The image sensor size is 1/3.2" with pixel size of 1.4 μm.For video recording, it can operate at 1080p at 30 frames per second.Hence, in the proof-of-concept demonstration, the transmitted signals are packet-based; and each packet is transmitted successively for three times in order to guarantee a complete packet is received in an image frame.Figure 3 shows the structure of the data packet.The channel gain matrix H can be obtained from the two training signals.For the training signal 1, the R, G, and B LEDs will emit signal in turn, and each transmitting a 11-bit training signal.This is to make sure only monochromatic signal can be received.For the training signal 2, the RGB LEDs will emit 11-bit training signals simultaneously.It is worth to method that the training signal 1 and 2 are used at the beginning for building up the VLC link.The payload performance can be continuously monitored in the mobile-phone; if the VLC performance decreases due to the link condition changes, the feedback signal could be sent from the user's side (mobile-phone) to the LED Tx side to request sending the training data again for re-establishing the transmission link.The data payload is 32-bit, and the training signal 1, 2 and the data payload are in on-off keying (OOK) format.In order to easily identify the header signal, the header consists of 4-bit periodic Manchester coded signal.This is enough for our clock recovery.In the demodulation process, the color rolling shutter pattern including many color fringes shown in Fig. 2(b) will be first separated into three different rolling shutter patterns based on the color filter of the iPhone5 as described in Fig. 1(c).Then each image file will be converted into grayscale format, in which 0 represents completely darkness while 255 represents completely brightness.A column matrix of pixel will be selected in each grayscale image frame.We have proposed and demonstrated using a second order polynomial fitting to select a column pixel in an image frame in order to mitigate the "blooming effect" as discussed in detail in [12].After the column matrix selection, a matrix of grayscale value having 1080 elements can be obtained for the R, G, and B colors.However, severe ICI will result due to the wavelength mismatch between the LEDs and the color filters as discussed above.Then MIMO described in Section II is used, and Figs.4(a)-4(c) show the data pattern of R, G, and B grayscale values respectively after the MIMO demodulation.High ER fluctuation can be observed.Then a second order polynomial fitting (black curve) is applied for smoothing.In the smoothing process, the grayscale values greater than the polynomial equation are assigned equal to the values of that equation; while the grayscale values smaller than that equation are assigned equal to zero.Hence, the ER fluctuation in grayscale data pattern is significantly reduced, as shown in Figs.5(a)-5(c) respectively.After this, another second order polynomial fitting curve can be constructed for thresholding.The thresholding is to distinguish the logic 1 and 0; hence the BER can be evaluated.In Figs.5(a)-5(c), as the header is Manchester encoded, much narrower periodic data can be easily distinguished from the payload OOK data.By estimating the frequency of these narrower data, the clock and sampling rate for the payload data can be obtained.If the LED source only shines part of the image sensor, the other part of the image sensor cannot reveal the bright and dark fringes due to low optical signal received.In this case, more complicated signal enhancement scheme should be used, such as using the Sobel method to enhance the signal [12].However, the processing complexity is increased.

Experiment, results and discussion
Finally, the BER performance of the RGB VLC transmission without and with the MIMO technique is shown in Fig. 6.When using the 32-bit payload, the effective data rate (by removing the header and duplicated payload) for the RGB transmission is about 3 x 30 frame/s x 32 bit/frame = 2.88 kbit/s.Besides, we can observe from Fig. 6 that the BER is significantly improved by more than one order of magnitude when MIMO is used (satisfying the FEC; BER = 3.8 x 10 −3 ), owing to the mitigation of the ICI.In Fig. 6, the blue channel performs the best when compared with the red and green channels because the blue color filter of the image sensor allows higher transmission; hence the blue channel has better sensitivity.As also shown in Fig. 6, a relatively high received illuminance is needed here due to no suitable focusing lens is used.The total included angles (total angle at which 90% of total luminous flux captured) are > 140° for the R, G, and B LED chips.In this proof-ofconcept experiment, only single R, G, B LED is used without proper lens for focusing; hence the transmission distance at ~500 lux is only <10 cm.Using a RGB LED array or focusing lens can increase the transmission distance.The BER performance is mainly determined by the decoding of the image frame received by the CMOS image sensor.Improved BER can be obtained by correctly decoding every bright/dark fringe in an image frame; and this can be achieved by increasing the illumination (putting the Rx closer to the LED or using focusing lenses) or decreasing the data rate (so that more pixels are used to represent each bit).

Conclusion
In this work, RGB LEDs were used to increase the VLC transmission capacity by WDM; and the color image sensor in mobile-phone was used.However, due to the wide optical bandwidth of the color filters, there was a high spectral overlap among different channels, and a high ICI happened.Here, we proposed and demonstrated a RGB VLC transmission using CMOS image sensor with MIMO to mitigate the ICI and to demodulate the rolling shutter pattern.A second order polynomial fitting was proposed and applied to enhance the data pattern ER, and then another second order polynomial fitting was applied for thresholding.BER performance was significantly improved when MIMO was used.
Figure 1(c) shows the RGB signals received by the color-insensitive sensing elements via the color filters.

Fig. 1 .
Fig. 1.Proposed architecture of the RGB VLC transmission using MIMO.(a), (b) Sketch of the optical spectra of the LEDs and Rx, (c) RGB signals received by the color-insensitive sensing elements after passing through the color filters.

Figure 2
Figure2shows the proof-of-concept demonstration of the RGB VLC transmission using MIMO to demodulate the rolling shutter pattern.The three independent data signals are programmed using Matlab, which is then transferred to two arbitrary waveform generators (AWGs) to perform the digit-to-analog (DAC) conversions.The two AWGs used have the sampling rates and bandwiths of 2 GSample/s and 240 MHz (Tektronix, AFG 3252C); and 50 Msample/s and 20 MHz (Agilent 33220A) respectively.The three data signals are then applied to the R, G, and B Txs (Lumileds Luxeon Z) as shown in Fig.2(a).The R, G, B and white (not used in this demonstration) LED chips are densely soldered on an aluminum starboard with spacing of 0.2 mm.The color chips used are "deep red", "green" and "royal blue"; and the "deep red" LED is based on AllnGaP material; while the "deep red" and "royal blue" LEDs are based on InGaN material.The RGB signals are then received by an iPhone5 Rx.Figure2(b) shows a typical RGB rolling shutter pattern showing different color fringes.Then, the recorded signal by the iPhone5 is demodulated by a Matlab program in a computer.

Fig. 2 .
Fig. 2. Proof-of-concept demonstration of the RGB VLC transmission using MIMO to demodulate the rolling shutter pattern.(a) RGB LED transmitter, (b) RGB rolling shutter pattern showing the different color fringes.

Fig. 3 .
Fig. 3. Structure of the data packet, including the training signal 1 and 2 for MIMO; and each packet is consisted of a header and payload.

Fig. 4 .
Fig. 4. (a)-(c) Data pattern of grayscale values (0 to 255) of the R, G, and B colors after MIMO demodulation with a second order polynomial fitting for signal smoothing.

Fig. 5 .
Fig. 5. (a)-(c) Data pattern of grayscale values (0 to 255) of the R, G, and B colors after smoothing, and with a second order polynomial fitting for thresholding.

Fig. 6 .
Fig. 6. (a)-(c) BER measurement at different illuminance without and with the MIMO for the R, G, and B colors.