Ultrafine and crosstalk-free 2D tactile sensor by using active-matrix thin-film transistor array

Abstract

Passive matrix (PM) technologies are widely used in various fields (e.g., manufacturing, human sensing, and robotics) to visualize the tactile pressure distribution. These technologies are powerful addressing methods with simple structures, low cost, and easy fabrication steps. However, crosstalk problems have been pointed out, especially in high-resolution fields. To prevent a crosstalk occurrence, we adapt active-matrix (AM) technologies for ultrafine tactile imaging. In this work, two-dimensional (2D) tactile sensors are prepared using AM arrays fabricated through standard display processes. Pressure-sensitive resistor sheets are then attached. The sensors have 6720 px in a 90 × 90 mm sensing area with a 1.1 mm pixel pitch. The crosstalk is evaluated by pressurizing the control area of the sensor and measuring the output in the non-pressurized area. No pixel is affected by the pressure outside the pixel itself, or no crosstalk occurs. For a demonstration, static pressure from soft toy balls and dynamic foot pressure during walking are loaded to the 2D tactile sensors. The differences in the contact mode by the ball type and the pressure of each finger are observed, thanks to the 1.1 mm-pitch without crosstalk. The 2D tactile sensors presented herein will contribute to the fundamental understanding of the contact interface and will have practical usage in sport sciences, biometric identifications, and tactile sensation of robots.

Introduction

Contact patterns and modes, such as pressure distribution, magnitude, and their change over time, are the fundamental physical information derived when two objects come into contact. Tactile distributions are useful in many fields, such as occlusion in industrial facilities, foot pressure in sports, and e-skins for co-operational robots communicating with humans [1, 2]. To realize two-dimensional (2D) tactile sensors contributed to these fields, the following are desired in addition to high accuracy: a broad size to cover the contact surface; a thin form that does not affect contact; an ultrafine resolution to recognize the contact pattern; and flexibility and lightweightness. In human sensing, flexibility and lightweightness can prevent injuries caused by dropping, bumping, or breakage of sensor devices. Using pressure-sensitive papers is a convenient method of visualizing pressure distributions with only the maximum magnitudes at each position. Strain gauges made of metal and other materials are effective in high-accuracy and -frequency pressure (force) measurements. However, making 2D sensors with a matrix of strain gauges that are broad, lightweight, and flexible is difficult [3].

Conventional 2D tactile sensors using pressure sensitive resistors (PSRs) with passive-matrix (PM) technologies have been proposed and widely put into practical uses [4,5,6]. The PSR resistance changes with pressure; hence, the pressure can be measured by using the PSR resistance. Resistance modulation is mainly caused by two methods, bulk or contact resistance modulation. Bulk resistance modulation is caused by the mutual spacing of conductive fillers in an insulating polymer film. On the other hand, contact resistance modulation is caused by the change in contact area between a conductor and an electrode. Figure 1 presents the PM technologies applying voltage at each intersection of the vertical (A–C) and horizontal (1–3) electrodes as a pixel. These circuit substrates consisting of vertical/horizontal electrodes and sandwiched materials are called PM arrays. PSRs are selected as sandwiched materials. Tactile distribution images are created from the resistance of each pixel. However, leakage and bypass currents are well known to exist in PM arrays, consequently causing crosstalk problems [7, 8]. In leakage, the current is flowing where it should not, while in bypass, the current is flowing in a path that is different from where it should be. As a specific example, consider the pixel B2 measurement in Fig. 1 by using the potential difference between electrodes 2 and B. A potential gradient is also generated between electrodes 2 and 3 if no pressure is loaded to pixel B2, and it has high resistance, and if pixel B3 is under pressure and has low resistance. The PSRs are seamlessly and uniformly sandwiched between the top and bottom electrodes as a single sheet; thus, the leakage current flows through electrodes 2–3. In particular, if there are defects such as foreign matter or agglomeration that cross over electrodes 2–3, they can easily become a path for leakage current. The crosstalk finally occurs, as shown by the dashed lines in Fig. 1. This makes it difficult to achieve an accurate measurement. In a higher resolution, keeping adequate gaps to suppress the leakage currents between the electrodes is more difficult. Crosstalk currents can also flow easier. As another possibility, when pixels B3, C2, and C3 have a low resistance, the bypass current, including the reverse current at C3, can flow through the pixels without the leakage currents (dotted lines, Fig. 1). The problem becomes more serious because the crosstalk may occur due not only to the neighboring pixels, but also due to more distant pixels with more complex current paths. Accordingly, great efforts are paid to prevent leakage and bypass currents, and ultrafine resolution is reported [9,10,11,12]. However, as a PM circuit, crosstalk is considered inevitable, and a drastic solution has not yet been proposed.

Fig. 1

Schematic diagram of the PM array circuit with horizontal (1–3) and vertical (A–C) electrodes. The intersection points are pixels. The PSR sheets are indicated as variable resistors at each pixel. The gray dashed lines depict a crosstalk occurrence with the leakage and bypass currents. The dotted lines show another crosstalk occurrence with only the bypass current

Ishikawa and Shimojo propounded epoch-making 2D tactile sensors with the transistors directly connected to the pixel electrodes to eliminate the unnecessary leakage and bypass currents [13]. This driving method involving active-matrix (AM) technologies is used in ultrafine flat panel displays (FPDs), such as liquid crystal and organic light-emitting diode displays, suggesting that ultrafine and crosstalk-free 2D tactile sensors can be achieved using FPD technologies. However, conventional FPDs are fabricated on rigid glass substrates; hence, only rigid tactile sensors can be realized. These rigid sensors have high breakage risk and lacks usability, making it difficult to apply FPD technologies to 2D tactile sensors as they are. Although flexible and crosstalk-free 2D tactile sensors are made with AM arrays having organic semiconductors on thin plastic films [14], organic semiconductors are not suitable for micro/nanofabrication; hence, ultrafine distribution sensors are hard to realize.

We recently developed flexible displays and light sensors through standard display fabrication processes with low-temperature polycrystalline silicon (LTPS) by using laser liftoff (LLO) techniques [15, 16]. LTPS has a higher reliability than an organic semiconductor; thus, sheet displays have already become commercially available. Circuit layers are lifted off from glass substrates by laser illuminations. This study demonstrates ultrafine and crosstalk-free 2D tactile sensors.

Sensor configuration and fabrication

Figure 2a shows the AM circuits used in this work. Thin-film transistors (TFTs) were placed at each intersection of the vertical and horizontal lines. These vertical and horizontal lines are referred to herein as the source and gate lines because they are connected to the source and gate terminals of the TFTs, respectively. The drain terminals were connected in series to the pixel electrodes, PSRs, and ground electrode. For the pixel B2 measurement, a constant voltage was first applied to gate line 2 to turn on the TFT. Another voltage was then applied to source line B. Consequently, only pixel electrode B2 transitioned to a high potential, and the current can through the PSRs to the ground. Current monitors are inserted to the ground, the resistance modulation can be measured by the current flow and the voltage applied to the source line. During this time, the OFF-state TFTs and the unselected source lines possessed a sufficiently high resistance for electrically isolating the other pixel electrodes from the current flow. At a given measurement timing, voltage can be applied to only one pixel electrode, where both the gate and source lines are selected. This enables crosstalk-free measurements in an ultrahigh resolution. This scanning method is in accordance with FPDs, but there may be a more optimal scanning method for tactile sensors.

Fig. 2

Overview of the AM arrays in this study. a Schematic diagram of AM array circuit. TFTs are installed at the intersection of the gate and source lines. The PSR sheets are indicated as variable resistors at each pixel. b Cross-section view of one pixel at the AM arrays. Glass substrate is replaced to PET film. c Top view of the AM array, showing a single pixel area with a dotted rectangle. One pixel consists of four rectangular sub-pixel electrodes. A ground electrode surrounds the sub-pixel electrodes. The insertion bar indicates 200 µm. d The pressure-resistance curve of typical SPRs. e A photo of flexible AM array substrate in this study

Figure 2b, c depict a cross-sectional view and a top view of one pixel of the AM arrays for the 2D tactile sensors in this work. Table 1 lists the sensor specifications. An approximately 3 μm-thick circuit layer was fabricated through standard display fabrication processes (e.g., chemical vapor deposition and photolithography) on a glass substrate coated by a 15 μm-thick polyimide layer. The circuit layer on the polyimide layer was lifted off from the glass substrate with laser ablation at the polyimide/glass interface during the LLO process. The lifted TFT substrate was then laminated onto a 100 μm-thick polyethylene terephthalate (PET) film as a stiffener. Finally, a PSR sheet based on contact resistance modulation and made from conductive past was attached on the TFT substrate. The PSR sheets were prepared in a difference of sensitivity with a total thickness ranging from 50 to 100 μm and coated on a PET film. The pressure-resistance curve of a typical SPR was measured and shown in Fig. 2d. Overall, the 2D tactile sensors were very thin (typically 250 μm thickness), flexible, and unbreakable [15, 16]. The circuit had 80 gate lines and 84 source lines in a 90 × 90 mm sensing area, with 6720 TFTs at the intersections. The substrate outline was 100 × 96 mm (Fig. 2e). The frame rate in this study is approximately 100 Hz.

Table 1 Specifications of the tactile distribution sensor used in this study

In active matrix arrays, size and resolution are independent, and this sensor specification is a kind of demonstration with reference to the manufacturing process [15, 16], previous research [12, 14] and products [17]. And also, the spatial resolution is no relying on the frame rate.

Results and discussion

To observe the sensitivity curves and the crosstalk conditions, 1–35 N static force was loaded and unloaded step-by-step with a force gauge (IMADA, ZTA-500N) in the center of the sensor. Silicon rubber was put at the endpoint of the force gauge as a pusher (5 × 5 mm area, 10 mm height). Pressure can be determined as 0.04–1.4 MPa from the force and the area of the pusher. Figure 3a, b depict the resulting image and the enlarged view, respectively. The pressurized area is presented as a 6 × 6 px square, indicating that the contact size of the pressurized pusher ranged from 4.4 to 6.6 mm and was in good agreement with the pusher size. The average of 4 × 4 pixels, excluding the 20 pixels at the edges, was calculated to evaluate the crosstalk conditions. Similarly, the unpressurized area of the entire sensor surface was divided into 181 squares of 4 × 4 pixels. The average was also calculated. The areas for which the average is calculated are illustrated by the gray drawings in Fig. 3b. In Fig. 3c, a gradually changing sensitivity curve was obtained only in the pressurized 4 × 4 pixels square. The other squares were almost zero in flat, completely unaffected by the pressure loading. In other words, crosstalk-free 2D tactile sensors were realized using the active-matrix thin-film transistor arrays. These results suggest that the AM arrays are an essential solution to preventing the crosstalk on 2D distribution sensors. Focusing on the pressurized square, a smooth graded sensitivity curve was obtained as shown in Fig. 3c. This result indicates the possibility of back-calculating the pressure at each location. Although a slight saturation tendency was observed above 1 MPa, the sensitivity range was sufficient for human measurements, as will be shown in the subsequent demonstrations. Pressure range adjustment requires the preparation of PSR materials, which should be studied with specific applications in view. Meanwhile, hysteresis, which is the gap between the loading and unloading processes, was observed in the sensitivity curve and may be attributed to the incomplete recovery of the PSR sheet components and the viscoelasticity of each material [18, 19]. Hysteresis causes poor reliability sensing; hence, further study is needed for sensor materials, measurement equipment, and data analysis method.

Fig. 3

Results of the sensitivity curve and crosstalk conditions. a Tactile image of this sensitivity measurement. b Enlarged view of the tactile image. The gray lines indicate the rectangles for division into 182 units. c Sensitivity curve of the 2D tactile sensors. The filled and open circles indicate the loading and unloading reactions at the pressurized square. Inset: other 181 squares

A 65 mm-diameter air-filled rubber ball and 70 mm-diameter foam-filled foam balls (Fig. 4a) were pressed against the sensors to demonstrate tactile imaging at a contact interface that was difficult to observe externally with static pressure. We considered two viewpoints here. How the mode, and the pattern of contact are imaged by the sensors. At a glance, the differences in the contact mode were observed between the rubber (Fig. 4b) and foam (Fig. 4d, e) balls. The foam balls showed a flatter and more uniform pressure distribution compared to the rubber ball, of which presser was concentrated at the outer edge. The results suggest that similar soft objects had different contact modes. This difference probably came from the internal structure, that is, filled with air or some soft materials. The rubber ball (Fig. 4f) was hollow; hence, the rubber membrane could conceivably penetrate inside when pressed against a flat surface. By contrast, the foam balls were filled with foam, making a flat contact mode without concaving into the inside (Fig. 4g).

Fig. 4

Result of the static pressure by the soft toy balls. a Photo of the rubber and foam balls pressed against the sensor. The subscripts correspond to each image. Tactile distribution image of the (b) rubber ball, (c) foam basketball, (d) foam tennis ball, and (e) foam baseball. Schematics of the contact modes of the (f) rubber and (g) foam balls with flat surfaces

The second perspective focuses on the surface patterns (structures) on the balls. We had three types of patterns here: concave and convex patterns on the foam balls and convex patterns on the rubber ball. In Fig. 4c, d, the approximately 3 mm concave structures that looked like basket and tennis balls were clearly observed as low-pressure areas in the images. In contrast, the convex patterns that imitated baseballs might not be found on the tactile images in Fig. 4e. When the balls were pressed against the flat surface, the convex structures came into contact first, and the pressures were concentrated there. The convex structures were then suppressed and flattened by the surfaces and the balls itself, making them difficult to observe in the pressure distribution image. In the case of the concave structures, a large area was in contact, and the pressure was broadly superseded, such that the concave structures were not completely pushed out by the pressure and would appear on the pressure distribution images. Although the effect of the shape must also be considered, the concave structures seemed tougher against presser suppression than the convex ones, which greatly affected the contact patterns. Seams on the rubber ball were also observed in the images (bright line in Fig. 4b). The contact mode session described that the rubber membranes tended to dent into the inside and did not seem to crush the seams. In summary, even similarly soft toy balls exhibited a great difference in the contact modes and in how the uneven structures appeared on the tactile images. The crosstalk-free 2D tactile sensors were needed to recognize and discuss the fine and minute differences in the contact interface that were difficult to observe from the outside.

This section demonstrated the benefit of the crosstalk-free 2D tactile sensors for practical studies. Practical research requires comprehensive data acquisitions; however, mounting small discrete sensors in all necessary locations is not feasible. Conventional distribution sensors are a good solution to obtaining comprehensive data as images, but crosstalk makes the data unreliable. In this work, we placed a 2D tactile sensor on the floor and stepped on it to imitate situations in sports science or rehabilitation engineering. Figure 5a shows the series of footprint pressures. The pressure imaging was performed at a rate of 100 Hz and the video was extracted approximately every 65 ms to make a frame-by-frame image. Thanks to the 1.1 mm-pitch resolution without crosstalk, both the foot outline and each finger were recognizable. Any finger or part can also be measured in terms of the contact pressure, area, center of mass, and their change in time, suggesting the possibility of not needing to worry about which positions on a floor to place the discrete sensors and not needing to adjust them depending on the subjects. Figure 5b depicts the changes in the averaged pressure and the contact area of the thumb. The trend of the averaged pressure and the contact area was divided into three phases, as indicated by A–C in Fig. 5b. In the initial phase A, the averaged pressure increased, while the contact area remained almost flat or slightly increased. In the middle phase B, the contact area switched to a decreasing trend. The averaged pressure still increased and reached the maximum. In the last phase C, the averaged pressure quickly decreased. The phase A tendency may possibly be caused by a shift in the center of mass from the left foot to the right foot and from the heel to the toe. The change at phase B can be assumed as caused by the foot kicking the floor with its full body weight. The last phase, phase C, corresponded to the transition from the stance to swing phases in the gait patterns. The results of this demonstration for practical studies have shown that broad AM array sensors can reduce efforts of mounting many discrete sensors onto the floor and can relieve worries related to the crosstalk problem. The AM technologies can also provide ultrafine resolutions for recognizing each foot part or region.

Fig. 5

Results of the footprint pressure with walking on the sensor. a Series of tactile distribution images of a right foot through walking. b Contact area and average pressure of a thumb. Markers A to C indicate the starting points of each phase in a and b

Conclusions

This study fabricated flexible 2D tactile sensors through standard display processes with the LLO technique and demonstrated the advantage of AM arrays that clearly prevented the leakage and bypath currents from causing the crosstalk problem and provided ultrafine tactile distribution images. The tactile sensors were thin enough relative to the thickness of expected object of biometrics (e.g., finger, foot) and robotics (e.g., foodstuff, daily necessaries), making them advantageous for discussing the contact mode and pattern at the interfaces between two objects. The 1.1 mm-pitch resolution enabled the recognition of each finger in dynamic foot tactile imaging, allowing the analysis of data at any location of choice on a posteriori basis without concern for crosstalk occurrence. An LCD with 3.76 µm sub-pixels has already been reported [20], and it is assumed that a tactile sensor with a similar pixel pitch can be designed. However, this pixel pitch is smaller than the thickness of the SPR sheet, and the sensor sheet itself may blur the pressure distribution. In other words, the minimum pixel size of active-matrix tactile sensors may be limited by the thickness of the sensor sheet. The pressure range and the hysteresis must be studied for specific applications. In addition to exploring sensor materials, a detailed evaluation of the sensor characteristics and the development of analysis methods appropriate for the application are also needed. Also, It should be noted that accuracy, reproducibility and dynamic range need to be addressed in future studies. The 2D tactile sensors were thin (typically 250 μm thick) and flexible; therefore, the pressure distribution on curved surfaces (e.g., handrails and grips) can be measured. We believe that the benefits of crosstalk-free sensors are not limited to the simple improvement of the resolution for recognition and identification by appearance. High-resolution tactile images are compatible with image processing and recognition technologies mainly developed for optical cameras with deep learning. For example, biometric identification by foot pressure distributions has been proposed [21, 22], but its identification capability is currently not as high as that of other authentication techniques. Ultrafine and crosstalk-free 2D tactile sensors enable the provision of stricter securities for walk-through authentications. Using thin broad sensors embedded in floors, handrails, and seats, early diagnosis can also be imperceptibly expected by walking, gripping, and sitting, respectively, during daily monitoring. The application of these thin sensors in e-skins has also attracted attention in the robotics field. Ultrafine and accurate tactile sensors for grippers will make robots more dexterous [23], while broad and flexible e-skins for bodies will offer novel communication between humans and co-operational robots.