Capacitive User Tracking Methods for Smart Environments

Passive positioning for smart environments, that could enable the tracking of people unobtrusively without any conscious human interaction and without compromising the user privacy with video based solutions, has remained as a challenging problem for decades [2]. Indeed, many of the current technologies are not welcome in a domestic environment where people should be able to live in comfort, feel safe and relax. For example, peoplemight find the positioning sensors too bulky and non-decorative, fear that their privacy is violated through them, or simply see no benefits of using them. Thus, these views pose serious challenges to the positioning technologies needed to enable the smart environments of the future.


Introduction
Passive positioning for smart environments, that could enable the tracking of people unobtrusively without any conscious human interaction and without compromising the user privacy with video based solutions, has remained as a challenging problem for decades [2].Indeed, many of the current technologies are not welcome in a domestic environment where people should be able to live in comfort, feel safe and relax.For example, people might find the positioning sensors too bulky and non-decorative, fear that their privacy is violated through them, or simply see no benefits of using them.Thus, these views pose serious challenges to the positioning technologies needed to enable the smart environments of the future.
Despite the presented problems, some specific passive user tracking technologies have greatly advanced in the last few years.Among these, capacitive user tracking techniques have established their position and have lately been shown to be usable for tracking people in three dimensions and for recognizing user contact with the objects of the environment.Therefore, this chapter will introduce four different capacitive tracking methods in sections 5-8 that can be used to realize passive and non-intrusive tracking of the human body.The presented methods, developed by Valtonen et al. in publications [47]- [51], enable together the creation of unobtrusive and user-friendly smart spaces, because they can be implemented in a non-distracting or unnoticeable way.Hence, the inhabitants of these environments can be supported in their daily tasks and concentrate on their primary goals rather than be burdened by the sensory actions of the environment.
Specifically, we at Department of Electronics, Tampere University of Technology 1 (TUT) have developed capacitive sensing methods that are able to reason human 3D position, body movement, posture and contact with the objects of the environment.The presented measurement methods and techniques are based on electric fields and on measurements of varying capacitances between the user and a set of sensing electrodes.By hiding the electrodes at least partly in the environment, the sensing can be realized in a unobtrusive way promoting the Mark Weiser's vision of calm technology [54].Furthermore, the privacy of the observed persons is not violated by extracting visual information of the subjects.Most of the developed methods have been evaluated in multiple short and long-term tests in the TUT Smart Home [8,10,51] and have been shown to be a feasible solution for their intended tasks.
To illustrate the importance of passive user tracking techniques for smart environments, we may consider a few examples of different types of applications.First, the developed methods could be used in smart homes to control the environment proactively and autonomously.By turning off unneeded devices or appliances, and by adjusting the heating to an comfortable level based on the user position and sensed context, the energy expenditure of homes can be pushed down [2,21,29].Second, in the health care sector user mental and physical activities could be remotely monitored by observing the user interactions with the home to support either the rehabilitation progress after an injury or the nurturing of the elderly [6,20,23,39].Third, in social media, user position and activity information could be automatically communicated to friends and relatives to make people more aware how their close-ones are doing [14,31].Finally, inhabitants of smart homes could self-evaluate and adjust their lifestyle based on the vast amount of information gathered, if it is processed to an easily digestible form [28].

Related work with other technologies
Video cameras have long been used for recording body movements and for positioning purposes [19,52].As a result, efficient algorithms for detecting human body movements from a video stream have been available for some decades.Although video-based motion capture is very efficient today, it has a major drawback with regard to user privacy.The simple fact is, many people do not want video cameras to be installed in such a personal space as their own home [46].Even if the video stream could not be fed outside the monitored room and would only be used, for example, for context analysis, the slightest risk of hacking and intervention by an outsider prevents most people from accepting cameras in such places.That is why there has been so much research into real-time location systems (RTLS) [18] and other alternative, wireless, positioning and activity sensing methods for monitoring people in indoor environments [3,16].These include ultrasonic [24], radio frequency [37,38,44], optical [9,22] and inertial-sensor [15,55] based techniques that require the person to actively carry a device.However, from the occupant's point of view these techniques are neither practical nor user-friendly because the user must always carry a tag or small electronic device on their person.Moreover, visitors cannot be located in the home without first equipping them with the necessary tags.Therefore, lately there have been several studies of passive or tagless methods for positioning the person indoors without the violation of privacy caused by visually-based solutions.One interesting approach was taken by Shwetak Patel et al. in 2008, when they published their building-ductwork-based sensing method for determining room transitions [27].The idea in this study was to utilise the existing ductwork infrastructure of the central HVAC systems found in many homes and detect the disruptions in the airflow.Even though the idea is intriguing, the method cannot be used to accurately detect movements within a room, and even room-to-room transitions can only be detected with 75-80% accuracy.Some years before that, in 2003, Susanna Pirttikangas et al. presented their methods for passive positioning and user footstep identification with a pressure-sensitive floor in [30].
They proposed the use of a novel Electromechanical Film (EMFi) material [26], which would produce voltage signals during pressure changes when stepped on.By placing 30-cm-wide strips of EMFi material orthogonally on the floor, they were able to calculate human X-Y position on the floor and identify human footsteps with a commendable accuracy.Later, in 2008, Jaakko Suutala and Juha Röning refined the original footstep identification methods and demonstrated a 95% classification rate with their floor system [45].Although, these results are significant and have been proven in practice, their systems cannot recognize stationary people because of the properties of EMFi material.Thus, they cannot be used effectively for detecting, for example, a person standing still on the floor or an elderly person who may have fallen to the floor.
Meanwhile, Yoshifumi Nishida et al. published in [24] a feasible alternative for passively measuring a person's position and posture in indoor environments.Their method is based on an array of ultrasound transmitters and receivers placed in the ceiling at intervals of around 18 cm.The pilot system they developed is able to position the subject three-dimensionally within the vicinity of the sensors and can calculate the head position both horizontally and vertically with about 5 cm accuracy.Although that system gives promising results in terms of accuracy, it requires the installation of hundreds or even thousands of sensors in the ceiling, if whole apartments or houses are to be covered by the system, which is not a practical solution for tracking the users.
In 2010, Daniel Hauschildt et al. published an infrared-based positioning system [4] that operates with passive thermopile arrays placed in the corners of the monitored room.The thermopiles detect the thermal radiation of human beings and enable the tracking of either one or two persons in a 30 m 2 room with a maximum error of 26 or 68 cm, respectively.Although their system is reasonably accurate for general indoor positioning, it is prone to reflection and dynamic background radiation effects that cause the sensors to give out false readings.

Capacitive sensing theory
Capacitive sensing is based on measuring the capacitance between two or more electrodes or physical objects.In human tracking applications, these objects have typically been the human body and the grounded environment, but in contrast to this common measurement configuration, this chapter concentrates on methods requiring both a transmitting and receiving electrode.Independent of the used configuration, to measure the capacitance in question, an electric field is artificially created around some sensing electrodes which are then used to conduct a displacement current into or from the measuring circuitry through the electrodes.Usually, the measurement is performed with a low frequency signal (about 30-100 kHz).This ensures pure capacitive coupling between the electrodes and also prevents magnetic fields from forming in the environment [36,51].
The electronics needed for capacitive measurement can consist of a signal generator, a phase shifter and a simple synchronous detector as shown in Figure 1.Alternatively, commercial capacitance measurement chips can be used as in publications [47]- [51], where commercial Analog Devices AD7746 [1] sigma-delta-based capacitance-to-digital converters (CDCs) [25] were used.Whichever implementation is used, both provide comparable results (see section 9.1) and the main differences are in the ease with which the measurement circuitry can be modified and in the number of components required.

Sensing modes
Three primary sensing modes can be identified and used for measuring body position in relation to the electrodes: transmit mode, shunt mode, and loading mode [42].These modes are visualized in Figure 1 with an example implementation of the measurement system.The measurement circuitry of Figure 1 always measures the serial capacitance of C t and C r in transmit and shunt mode by analyzing the displacement current i r flowing from the transmitter to the receiver.In loading mode, however, only the displacement current i t flowing from the transmitter through the body to the ground is measured and the capacitance between the electrode and the body is calculated.

Transmit mode
The sensing mode used here is the transmit mode, when either the capacitance between the transmitting electrode and the human body (C t ) or the capacitance between the receiving electrode and the human body (C r ) is much greater than the capacitance between the human body and the ground (C g ).In practice, these situations occur when the measured person touches or is very close to either of the two electrodes.In the transmit mode, the body is capacitively coupled to the nearby electrode and the potential of the body changes in response to the potential of the electrode.In practice, a person usually increases the electrode area by a significant amount and reduces the observed distance between the electrodes.Therefore, the measured serial capacitance of C t and C r is greater when a human body is close to one of the electrodes than when there is no body nearby.This sensing mode is The floor is constructed from large 60×60-cm raised floor tiles and the transmitting electrodes are attached to the underside of the tiles.The system's receivers are placed in the environment, for example, under the top of the table.[51] used in all publications [47]- [51], because it allows the electrodes to be located in a practical way that enables the distance between the user and the electrode to be measured.

Shunt mode
In shunt mode, the capacitance to the ground through the human body (C g ) is considerably higher than the capacitances C t or C r between the measurement electrodes and the human body.Here, the human shunts the electric field to the ground and prevents the emanated displacement current from being detected by the receiver, thus decreasing the measured serial capacitance of C t and C r .However, the measured serial capacitance of C t and C r depends largely on the capacitive coupling between the measured body and the ground.Indeed, if the grounding of the body is effective and C g is high, the measured serial capacitance of C t and C r is low and vice versa.Nevertheless, by measuring the received displacement current i r at the receiver, the portion of the body between the two measurement electrodes can be deduced.If either C t or C r are equal to C g , the measured serial capacitance of C t and C r in a measurement system is about the same as it would be without the presence of a human body.

Loading mode
In loading mode, only the displacement current i t flowing from the transmitter through the body to the ground is measured and the capacitance between the electrode and the body is calculated.This mode is often used because, apart from the surrounding ground, only a single electrode is required to make the measurement.However, as with the shunt mode, the coupling between the human body and the ground affects the resulting measurement result so it is hard to measure the absolute distance between the body and the electrode.

Capacitance model
Based on the above description of transmit mode sensing, we can develop a simplified capacitance model that describes the measured electrical properties of the test systems that were constructed in publications [47]- [51].This simple capacitance model is shown in Figure 2 with a cutaway picture of the TUT Smart Home used as the test environment, with the system presented in publication [51].The following paragraphs explain the terms used in this capacitance model.

Feet capacitance C t F
The capacitance model incorporates two capacitances that are formed between the electrodes and the human body.First, C t F is the capacitance between the human feet and transmitter t.C t F always increases when the common area between the person and the transmitter grows, and vice versa.Likewise, C t F increases as the distance between the transmitter t and the body decreases.Typically, C t F ranges between 20-700 pF with the floor-tile types used in the test systems of publications [47]- [51] when shoes are not worn [47].However, if shoes are worn they insulate the person from the transmitting electrodes and thus act as an additional insulator.As a result, different types of shoes significantly decrease C t F and thus affect the measurements.In fact, C t F can be up to 96% lower if shoes with 3-cm-thick soles are worn [47].

Body capacitance C B
The second of the body-related capacitances is C B .It is formed between the body and the receiver(s) of the system, insulated by air, textiles, or wood.However, with multiple receivers, it is formed mainly between the body and the closest receiver, because the strength of the electric field is strongest near the body from which it emanates.Furthermore, because only a single receiver channel is used in all the implemented systems' hardware, the receiver signals cannot be distinguished from each other.Typically, C B varies from around 1 to 10 pF but can be significantly higher if the tracked person is in actual contact with the receiving electrodes [47,51].

User induced capacitance C t U
Because the electronic circuitry which we used measures the total capacitance between the transmitting and receiving channels, C t F and C B cannot be distinguished from each other in the acquired data, and so they must be combined into a single term.Also, because only a single receiver channel is used in our hardware implementations, these two capacitances are simplified in our model into one single capacitance C t U for any given user.Because C t F and C B are connected in series, C t U can be defined with the equation assuming that any stray capacitances in the environment are discounted.
Here, C t F increases as the area of the feet touching the floor increases and decreases as this area decreases.Likewise, C B increases when the person's body is brought closer to a receiver and decreases when the person recedes from the receiver.

Offset capacitance C t O
When nobody is in the apartment, there is an offset capacitance C t O between each transmitter t and the receivers.Because C t O is in parallel with C t U in the capacitance model, it affects the measured total capacitance C t TOT between a transmitter t and the receivers as in the equation To get a reference level with which all future measurements can be compared and to obviate the effect of C t O , this only needs to be measured once before the person enters the measurement space.Thereafter, this measurement can be subtracted from all subsequent measurement results to yield C t U on its own.
Capacitive User Tracking Methods for Smart Environments 7 Stray capacitance C t

S1
Capacitance C t S1 occurs between each transmitter t and the ground, depending on which parts of the environmental structure are acting as an insulator.With the transmitters shown in Figure 2 in the TUT Smart Home, air acts as the insulator between the large bottom surface of the transmitting electrode and the ground.However, there is a significant capacitive coupling between the metal pillars and the ground, so the plywood spacers on top of the pillars are part of the insulating elements.In all of our test systems, because C t S1 between transmitter t and a receiver is only measured by gauging the received displacement current at the receiving electrode, the amount of displacement current flowing into the environment from the transmitting electrode does not affect the result as long as the signal source is able to drive the desired waveform into the transmitter [25].Thus, C t S1 does not affect the measured capacitance value with a buffered output present in our test systems' hardware, and so its effect on the measurement can be neglected.

Stray capacitance C S2
Capacitance C S2 is formed between the human body and grounded objects in the environment.An increased C S2 can cause the received displacement current to decrease, because C S2 conducts a small part of the displacement current emanated by the person to the ground and thus decreases the capacitance reading C t U .However, C S2 remains fairly constant even when large and grounded conductive objects are near the measured person [47].At most, these types of conductive objects increase C S2 only by some tens of percent.Because the absolute value of C t U is not of major interest and the error reflected in it is fairly constant, unless the person is leaning against a wall, C S2 can be neglected for position measurements.Likewise, because the appropriate levels for C t U must be calibrated to recognize user contact with certain household objects, C S2 does not have a significant effect on the activity measurements since its effects are calibrated at the same time.

Stray capacitance C S3
Capacitance C S3 occurs between the receivers and the ground.With a static receiver position C S3 remains almost constant at all times [47].Because of the internal implementation of the AD7746 CDC, it is theoretically insensitive to leakage current errors and parasitic capacitance to ground [25].However, due to the inevitable imperfections in the manufacture of the CDC, the receiver should always be positioned so that C S3 remains below 300 pF (see figures 9-10 in [1]) to prevent large errors in the results with the measurement configurations of publications [47]- [51].Because, when positioned correctly, C S3 has only a slight effect on the measured value of C t U , and also because the constant error caused by it is, after all, calibrated during the system initialization, it can be neglected during the practical measurements.

State of the art in capacitive user tracking
It is capacitive methods for human tracking applications that are attracting the most interest because 1) they provide a way of preserving personal privacy, 2) their sensing electrodes need not to be visible to the user, 3) they can be used to recognize both static and moving people, and 4) they can be implemented with large and cheap electrodes that can cover large areas.Indeed, as early as 1993, a simple electrode configuration for detecting the presence or movement of a person close to a robot was developed by Nils Karlsson et al. in [13], and later developed further in [11] and [12].The purpose of this system was to stop a robot moving to ensure the safety of any person who came too close to it.Two years later after Karlsson's 'person detector', Thomas Zimmerman et al. from MIT Media Laboratory presented various ideas on capacitive applications in [56], including a two-dimensional Finger-Pointing Mouse and a Person-Sensing Room.The floor of the Person-Sensing Room was covered by a transmitting electrode and four receiving electrodes were placed on the walls.Using the floor electrode, a measurement signal was capacitively coupled to the subject and the strength of the emanated signal from the subject was measured by the wall electrodes.The room was able to use this information to locate a person two-dimensionally on the floor plan.
In 1999, Joshua Smith from MIT Media Laboratory published his dissertation thesis [41] covering electric field sensing with different kinds of sensing applications.Later, some of their most advanced applications, three dimensional Field mice [40] and Gesture Wall [42], were presented in depth.The Field mice was able to capture hand position and alignment above the measuring electrodes with shunt-mode measurements using a single transmitting electrode and three receiving electrodes arranged on a plane.The Gesture Wall, in contrast, used the transmit mode to transmit the measurement signal through the human body to four receiver electrodes placed at the corners of a screen.The users of the system were able to draw on the interactive screen by making hand gestures in front of the screen.
Recently, Henry Rimminen et al. have studied the use of electric fields for positioning people over a segmented floor electrode [35,36].Instead of using the transmit-mode measurement technique, he proceeded with the loading-mode measurement method, which measures the capacitance between a transmitter and the ground.Specifically, his systems scanned the floor area with the floor electrodes and toggled each electrode to transmit one measurement signal at a time [36].When the electrode below the feet of the person was actuated, a larger than normal displacement current flowed from the body through the air to the surrounding grounded electrodes.The position of the person was determined using the physical locations of the electrodes and the level of current measured from each electrode.Although his systems achieved about the same level of positioning accuracy as our TileTrack system [48], the loading-mode measurement method used in his studies has not been demonstrated in user activity recognition applications, nor can it be modified to measure human height and multiple postures, as is possible with the transmit-mode systems presented in this chapter.Nevertheless, in [34] they demonstrated a fall detector that was able to reason when a person was lying on the floor.

Tile-based positioning
The test systems of publications [48], [50], and [51] use transmit-mode sensing to locate users at floor level.In contrast to the systems of Karlsson [13] and Zimmermann [56], the electrode arrangement in these studies consists of a segmented floor composed of multiple transmitters and one or more receivers placed in the vicinity of the transmitters.The segmented floor in all of these studies was built from raised floor tiles that had a conductive metal layer on the bottom surface of the tile (see Figure 2).Regardless of which receiver configuration was used, the same tracking algorithm was used in all of these studies.

Transmitter size
The test systems of publications [48] and [50] used 60×60-cm commercial raised floor tiles that had steel reinforcement on the underside of the tiles.These tiles, shown on the left-hand side of Figure 3, were also used in [51] with the large test system built into the TUT Smart Home.In addition, however, multiple custom-built tiles with roughly 30×30-cm transmitting electrodes, shown on the right-hand side of Figure 3, were used to achieve more accurate positioning results.Indeed, the smaller transmitters can be placed in places where more accurate position information is necessary or preferable.However, dividing the area of  [48], [50], and [51].The left-hand side of the figure shows the bottom surface of a standard, commercial 60×60-cm raised floor tile from the TUT Smart Home.The inset photo shows a snap fastener soldered to the middle of the electrode for cable connection and a screw drilled next to it to prevent the snap fastener from being flattened.The right-hand side of the figure depicts a custom-built floor tile with four roughly 30×30-cm copper foil transmitting electrodes.These electrodes are partly hidden by the two plywood boards on the sides that stiffen the tile and raise it to the same height as the commercial raised floor tiles.[51] a large transmitter into four smaller segments also makes the measurements from these tiles four times slower with the algorithms we used, and increases the amount of wiring needed to connect the tiles to the measurement circuitry.Therefore, when deciding what size floor electrode should be used, the assumed usage frequency and application possibilities of the installation environment should be taken into consideration.It is also important to note that there is no upper limit to the size of the transmitter, so the compromise between accuracy, speed and the amount of wiring required can be made freely according to which room or area needs to be tracked.

Receiver configuration
Both publications [48] and [50] used a vertical wire or a small plate electrode as a receiver but the large-scale test system of publication [51] relied only on receivers R1-R5 installed below the ceiling (R1-R2) or in common household items (R3-R5) as shown in Figure 4.Although the same algorithm was used in all of these studies and the slight variation in the receiver configuration does not affect the tile-based positioning method, it must be noted that all the used receivers must be installed close enough to the transmitters for the system to operate reliably.Based on the results of publication [48], the signal-to-noise ratio (SNR) decreases almost linearly according to the distance between the tracked person and the receiver.Therefore, in practice the receivers must be installed at a maximum of about three meters away from the transmitters to enable a good capacitive coupling between the transmitter and the person.This is also apparent from Figure 4, which shows the received signal strength in the TUT Smart Home as a percentage of the maximum signal.However, the measurement range of the receivers can be extended by ensuring that there is either no insulation, or only a thin layer of insulating material between the transmitter and the feet of the person, and by raising the emitted voltage.

Algorithm
In tile-based positioning, a measurement signal is driven into the tracked person from the transmitting electrodes in the floor and the received signal strength, i.e.C t U ,i s

Posture
Transmitter size (cm) measured by the receiver electrode.The measurements are taken in a time-multiplexed manner and a single tile measurement takes 11 ms.The 2D user position is calculated using the relative C t U values from each transmitter t according to the physical location of the electrodes.C t U increases when the feet of the person cover more of the electrode area, and, the greater the C t U , the closer to the centroid of the transmitter t the person is assumed to be.

Key results
The practical test systems that were implemented demonstrate that this type of tracking system can be installed in the structures of the building and at least partly hidden from the users.As stated above, the tracking accuracy of these types of systems depends on the size of transmitter used.Furthermore, a standing person can be located with greater accuracy than a moving person, because of the fact that there is better electrical coupling between the person and the transmitters when they are in a standing posture with both feet on the ground.Table 1 presents the measurement results with the two different sizes of transmitters measured without shoes.With shoes, the results can be expected to be somewhat poorer, although no exact measurements have been made [47].
When analyzing the results of Table 1, we see that the mean and standard deviation are almost the same for both transmitter sizes.Although the maximum errors differ significantly because of the weaker signal reception with the smaller transmitters, which have a thicker insulation layer between the transmitting electrode and the person, the smaller transmitters actually have about a 38% lower margin of error at the 90% confidence level.
As a part of the study presented in publication [51], a long-term test was performed in the TUT Smart Home to test the tile-based positioning system outside a laboratory environment.A test person was asked to live in the apartment for 14 days and his position was recorded while he was inside the home.Figure 5 shows a one-hour example of the data acquired during this test period on the 21 st of October, 2009 between 20:00 and 21:00 hours, with the raw positioning data obtained from the hardware with a little filtering and with a smoothed track which depicts more precisely the true path of the body of the person.It is easy to see from this figure that the raw data obtained with the tile-based positioning system is rather erratic because it follows the individual steps of the person.All the gathered positioning data was released for public use and is freely downloadable at [17].
The hit rate of this system, analyzed from this one-hour data sample, was found to be 95.8%.It was calculated by dividing the number of times the system had not lost the track of the 123 Capacitive User Tracking Methods for Smart Environments Cable connections Figure 6.Both the transmitters and the receivers can be hidden in household items in order to recognize user touch and proximity to them.On the left-hand side, the metal front surfaces of the refrigerator (left appliance) and freezer (right appliance) were used as transmitting electrodes to detect when these appliances were used in the TUT Smart Home.On the right-hand side, a copper foil receiver is attached under the tabletop of the sofa table.With this receiver, the measured data was used both to detect contact with the table surface and to position the user on the floor in the vicinity of the table.The connecting cable for the receiver electrode, going under the floor through the carpet, is also visible.[51] person, for at least a period of one second, with the total number of measurements performed.In other words, the system was not able to measure the person position for a total of about two and half minutes during the one hour sample.The places where the person was lost were distributed in the following way: a total of about one second in the dining room, 25 seconds in the hallway and two minutes and five seconds in the kitchen.In contrast, the false hit rate of the system, analyzed from this same one-hour sample, is 0.2%.This figure was obtained by calculating the number of times the system detected the person in the sensed floor area while the person was actually still in the bathroom or on the balcony, and then dividing that number by the total number of measurements performed during the periods of absence from the tracking area.In figures, the person was momentarily falsely recognized on the floor five times out of the 2404 measurements that were performed during the eight minutes and ten seconds of absence.

Contact sensing
The activities of people in a smart environment can be monitored through their interactions with the objects in the environment.For example, the system described in publication [51] can observe user contact with common household items such as a bed, sofa, table or refrigerator.When used in conjunction with the tile-based positioning method (see section 5) as in publication [51], the position of the person can be deduced simultaneously with the user interactions using a single measurement system.

Electrode configuration
To measure interactions with the environment, transmit-mode sensing can be used to recognize user touch to both transmitting and receiving electrodes that are hidden in the objects of the environment.For example, sensing electrodes can be installed in beds, sofas, tables or electrical appliances as in publication [51].The left-hand side of Figure 6 shows an example how the front doors of the refrigerator and freezer were connected to the measurement system in the TUT Smart Home and used for detecting when carried dishes to the kitchen, 5) checked the home's status from the bedroom computer, 6) undressed, 7) took a shower in the bathroom, 8) cooled off on the balcony, and 9) dressed in the bedroom.The walking paths during this one-hour period have already been shown in Figure 5. [51] they were used.In the case of non-conductive objects, thin copper foil can be applied to the item, such as to the sofa table in the TUT Smart Home shown on the right-hand side of Figure 6.

Measurement method
Human contact with household objects can be detected by measuring the C U for the electrode attached to the object.The magnitude of C U changes in direct proportion to the person's proximity to the electrodes.In other words, C U is high when the absolute distance to the electrode is small and vice versa.However, the changes are non-linear with the distance because the measured capacitances increase quadratically as the distances between the person and the electrodes become shorter.In the test system described in publication [51], any contact with the object is given as a binary signal that is formed by comparing C U to a predetermined constant threshold capacitance for the object.For example, the test system in publication [51] was defined to give a binary one when the person touched the object and a zero when the object was not touched by the person.

Key results
As a part of the study in publication [51], a two-week long test was performed in the TUT Smart Home to test the contact-sensing system.Figure 7 presents an example of the activities detected in the TUT Smart Home during a one-hour period below the annotated events that were recorded by the test user on a portable voice recorder.This diagram shows that the annotation data matches well with the recorded contact data.Moreover, if the plotted tracks of the person shown in Figure 5 during this same one-hour period are analyzed together with the contact and annotation data, they fit seamlessly together.Therefore, we can conclude that by embedding the electrodes of a tile-based positioning system in the household objects of a smart environment, the interactions with the environment can be detected with a good accuracy.Furthermore, when the positioning data is analyzed together with the recorded interactions it is possible to infer the activities of the person under observation within a particular context.

Electric field ranging
Human position, leaning and hand or leg movements can be captured by measuring the capacitance between a transmitting floor electrode and a receiver electrode close to the person, when the person stands or moves over the floor electrode.This section presents such a passive, simple, and affordable electric field based ranging method that transforms the physical input 125 Capacitive User Tracking Methods for Smart Environments Figure 8.The electric field ranging method allows even slight body movements in the tracking space to be detected.By measuring the user distance from the vertical wires placed in each corner of the tracking space, the user position can be calculated and used in addition to indoor positioning, for example, to control a flight simulator game.[49] into a two-dimensional position in an indoor environment.The method and a demonstration platform has been built and presented in detail in [49] and will be presented in this section with sufficient details about the system structure, capacitance model and results for the reader to understand the operation and possibilities of the measurement setup.Although this test system at the current state of development can only be used in small scale setups, it provides interesting possibilities to application developers, for example, in gaming, virtual exhibition, and virtual reality industries.
Instead of using multiple transmitters and a small number of receivers, it is possible to track a person with a single transmitter and multiple receivers.This is made possible by calculating the capacitance between the person and the receivers and converting it into distances.Moreover, this method allows even slight movements, such as leaning or moving an arm or a leg, to be discerned, assuming that an adequate signal can be received by the receivers.As was demonstrated in publication [49], this method allows the test system developed in publication [49] to be used not only for human position tracking, but also in gaming, virtual exhibitions, and virtual reality applications.

Construction
The test system described in publication [49] consists of a single large transmitter, 180×180 cm in size with four vertical wire receivers placed at the corners of the transmitter.The transmitter consist of nine commercial raised floor tiles that have been electrically connected, and the receivers are made of standard power-line cables.The whole test installation has been placed next to the structures of a virtual reality cave, which has two background projection screens beside the transmitter.Figure 8 shows the physical construction of the test system with an example application of flight simulator control.

Measurement method
This system measures the capacitances between the user, standing on a transmitting floor electrode, and four vertically aligned receiver wires placed in the corners of the tracking area.The system uses the transmit mode and measures one receiver at a time at 19.5 Hz.The measured capacitances are converted to absolute distances between the user and the receivers using a capacitance-to-distance conversion function in the horizontal dimension.If the distances between the user and the static receivers are known, a non-linear system of equations can be solved to calculate an unambiguous position for the user.Specifically, the system measures the value of C r U for each receiver r at a time.A predetermined capacitance-to-distance conversion function is used to convert the measured C r U value to an absolute distance value d r between the receiver r and the user.The non-linear system of equations is solved in real-time to find the best approximation for the 2D body position x by using all four of the computed distances d r from the receivers using an iterative least squares (Gauss-Newton) algorithm.Also, because the C U changes vary somewhat according to the person's height, feet size and footwear, the computed distances are corrected with a correction factor k whose value typically ranges between 0.6 and 1.5.Altogether, the measurement equation to be solved can be written as where, b r is the location of the receiver r and e r is the measured error for the receiver.

Key results
The test system which was constructed and described in publication [49] demonstrates that a person's body position and movements can be tracked passively with a simple electrode configuration and affordable electronics by converting the measured capacitances to absolute distances.The system works with different size people and in some applications the correction factor k could be used to adequately identify different-sized people or to identify the insulation properties of an individual's footwear.The test system can track a human body with relative accuracy of less than 10.4 cm and absolute accuracy of less than 15 cm. Figure 9 shows two different ways that the system positions the user in a flight simulator scenario.First, the inner circular shape shows the limits to how much a test person was able to change the tracking result by swivelling in different directions with his arms at his sides.Second, the larger radial results show how the test user was able to position himself when he stepped in 8 different directions and bent his body in the direction of the step.The figure reveals that the test person was able to move his computed position away from the center position by an average of about 15 cm when swivelling and 70 cm when taking the single steps.In addition to these experiments, this system was tested for mouse cursor control, virtual-object viewing-angle manipulation, and drawing.

Height and posture sensing
It is difficult to unobtrusively recognise a user's height and posture passively without video cameras.In publication [47] we demonstrated a method for distinguishing a person's posture with a simple measurement system.The measurement method is based on transmit-mode measurements and instead of calculating the horizontal distance from the person to the receiver as in [49], the conversion is done vertically and the distance between the user and a receiver above the user is determined.Further, by first measuring the standing height of the person, the user's posture and vertical head position can be determined with these simple techniques.The response of the electric field ranging method was tested in publication [49] with two postures.First, a test user stood in the middle of the tracking area and swiveled through 360 • while keeping both feet on the ground.Second, he stepped forward in eight different directions while keeping the other foot in the same place.Modified from [49].[47] set in a horizontal plane above the user.Figure 10 demonstrates the potential distribution in this type of a measurement setting in a well conducting building when the receiver has been installed at a height of 240 cm and the person standing on the transmitter is 170 cm tall.

Measurement method
The height of a person is calculated using the measured C U value by converting it to an absolute distance between the person and the receiver and by subtracting this from the height at which the receiver is installed.The conversion is done with a pre-calibrated capacitance-to-height conversion function that must be calculated for the operating environment before the system goes into operation.This conversion function can be derived either from simulation results by calibrating it for the given environment or by experimentally defining it in the operating environment.The head height of the person at any moment can be determined by calculating a percentual height h % of a person from the standing height h s with the equation Furthermore, this percentual height can be used to determine the posture of the observed person, for example, by using the posture classification presented in Figure 11.However, this classifier has not been verified, because the small number of test persons (n = 14) available in [47] were all used to create this classifier.

Key results
Both the simulation and experimental results from publication [47] shown in Figure 12 demonstrate that C U changes with the height of a user in a standing position.Moreover, since the simulation results and the actual results have a similar shape in Figure 12 but a somewhat different slope, the scaled simulation graph in Figure 12 shows how well the simulation and experimental results correlate with each other.This scaled graph was calculated from the simulation results using the equation where the constants have been experimentally defined. .Simulated and measured capacitance-to-height conversion functions with scaled simulation results from publication [47].[47] Statistically, the experimental system described in publication [47] was able to calculate the height of a standing person with at least 7.1 cm accuracy.In all other postures ranging from standing erect to crouching as low as possible, the absolute accuracy of the system was always determined to be 21.7 cm or less.At 90% confidence level these values were 5.2 cm and 14.3 cm, respectively.If the relative change in the user's height from a standing posture is enough for a given application, the height change can always be measured to within at least 14.6 cm accuracy and at 90% confidence level to within 9.1 cm accuracy.Furthermore, if the system is calibrated for a single individual, the absolute and relative accuracies for a crouching person are 11.6 cm and 4.5 cm respectively at 100% and 90% confidence levels.
These unobtrusive height and posture measurement methods can be used in many context and activity recognition applications.Indeed, in smart environments the activities of the tracked persons can be reasoned with higher accuracy if the three dimensional position and posture of the person can be inferred.For example, if we know that a person is sitting next to a table, rather than standing, we can infer that the person might be eating or reading something.On the other hand, if the person is standing, we can infer that they might be laying the table.Similarly, if people are sitting on the floor and playing a board game, we may distinguish that from people standing in a circle and chatting with each other.Or, if a person is exercising on the floor and doing some sit-ups, we can differentiate that action from someone just lying on the floor.Therefore, the elderly could be monitored to support independent living at home and raise the alarm if the elderly person should fall and not get up again.Finally, smart homes could offer some unique services to their inhabitants like proactive home appliance control and real-time context delivery to family members and friends outside the home.Furthermore, if there are sufficient differences in the heights of the family members, they could be distinguished from each other, and so individualised services could be provided for them through the home user interfaces and actuators.

Discussion
This unobtrusive way of measuring a person's position, posture and contact with the environment in 3D will be helpful in creating new smart environments that are calm and non-distracting to their users.Moreover, these methods can be used outside smart environments in many different types of applications.In this section, we will examine the pros and cons of the methods, compare the methods to the state of the art technology, and present ideas for future work.

Accuracy and reliability
The results presented here indicate that the methods which have been developed cannot achieve the same level of accuracy as can be obtained with the methods used in the state-of-the-art active systems which can be applied in smart environments.For example, the two well-known ultrasound-based location systems, Cricket [32] and Bat [53], can position a person with about 10 cm and 5 cm accuracy, respectively.Likewise, a commercial Ubisense location system based on UWB (Ultra-wideband) technology can track a person with about 15 cm accuracy [16].However, the test systems described in publications [48], [49] and [51] do demonstrate somewhat better accuracy than the the passive infrared system of Hauschildt et al. [4], which can only position a single person with an accuracy of 26 cm.On the other hand, our systems are not as accurate as the passive ultrasound system of Nishida et al. [24], which can track a human head with less than 5 cm error.However, it is important to note that in its current state of development the system of Hauschildt et al. suffers from reflection and dynamic background radiation issues, and the ultrasonic system of Nishida et al. would require the installation of hundreds of sensors in the ceiling of the TUT Smart Home in order to cover most of the rooms.
If the accuracies of the test systems presented in publications [48] and [51] are compared to the capacitive positioning system of Rimminen et al. [35,36], we have a slightly better mean accuracy than Rimminen's system.Indeed, Rimminen et al. use 25×50-cm electrodes on the floor and get a mean accuracy of 21.2 cm for people walking, while our systems obtained a mean accuracy of 9.4 cm and 17.5 cm with the 30×30-cm-sized and 60×60-cm-sized floor electrodes respectively.Furthermore, the standard deviations in our systems are also somewhat lower: 13.2 cm for Rimminen's system versus 6.5 and 11.3 cm respectively for the small and large transmitters used in our systems.The main reason that our systems are more accurate than the Rimminen one, even with the larger electrodes, is probably due to the fact that Rimminen et al.'s system had wiring between the electrodes on the floor surface as shown in [33, p. 11].This meant that the available electrode area on the floor surface was reduced, which led to an increase in the positioning errors.
The reliability of the tracking system built in the TUT Smart Home in [51] can be analyzed with the hit rate and false hit rate figures presented in section 5. To understand why the tracking system lost the track of the person in the TUT Smart Home, we must look at where the system lost track of the person.In section 5 we showed that, most of the time, the person was lost in either the kitchen (82.7% of the cases) or the hallway areas (16.6% of the cases).
If we compare this distribution with the Figure 4, we can see that the places reducing the hit rate have a relatively low received signal strength.Thus, a low signal strength greatly affects the reliability of the tracking system.In addition, the contact of the person with grounded sources in the kitchen, i.e. the dishwasher, the sink and the stove, cause the transmitted signal to couple with the ground so it is not received by the system.Therefore, for much of the time when the person was considered lost he was preparing food and touching one or other of these grounded objects in the kitchen.This is also very clear from the publicly released test data [17].
The acquired hit rate of 95.8% is slightly better than the 91.6% achieved by Rimminen at al. [35].Again, the reason for this difference is caused by the gaps between the floor electrodes in the UPM Kymmene Oyj sensor laminate used in their work [33].Furthermore, our hit rate is better than the maximum hit rate of 87% obtained with the pressure sensitive floor in the Gator Tech Smart House [7].Nevertheless, some ideas to further enhance the reliability of our system will be presented in section 10.

Construction, cost and complexity
The physical construction of the electrodes in all of the presented systems is fairly simple, because they consist of single layers of conductive material, which is easily available.For example, the floor electrodes could be built from aluminum foil placed underneath the floor surface, which could be either wood, plastic or ceramic.Similarly, the receiver electrodes could be built out of almost any conductive solids, textiles, or even from a sparse wire net.In fact, a 0.22 mm-thick wire net receiver was shown to be a feasible solution in publications [47] and [51] to create a barely noticeable receiver below the ceiling.
Unfortunately, the current design requires a single coaxial cable for each electrode, which means that a large amount of cabling is required to cover a house.In the case of the TUT Smart Home, about 400 meters of coaxial cable were installed to feed the measurement signals to and from the electrodes.One solution to this problem would be to replace the cabling and the electrodes with, for example, the thin sensor laminate made by UPM-Kymmene Oyj that was used in the work of Rimminen et al. [35,36] and is shown in [33, p. 11].This 150-µm-thick laminate was made from a 13-µm-thick aluminum film that was laminated between two layers of PET (polyethylene terephthalate) film.All the wiring to the 36×30-cm-sized electrodes were incorporated into this laminate using etching techniques and were made accessible at the edge of the laminate using a standard connector.
Based on the large-scale tile-based positioning system implementation of the TUT Smart Home described in publication [51], we can calculate the total cost per square meter for a real-world capacitive positioning system.If we include all the materials and devices used to construct the positioning and contact-sensing system of [51] but exclude the cost of labour and the PC used for position calculations, we get a total price of about 5800 e for the roughly 53 m 2 area covered by system.This equates to a cost of 110 e/m 2 , of which only about 16 e/m 2 goes on the electronics and cabling, and 4 e/m 2 on both the copper foil and silverized-textile receivers placed in the environment.Clearly, most of the costs arise from the raised floor tiles, which are relatively expensive, being about 108 e/m 2 and 68 e/m 2 for the commercial and the custom built floor tiles respectively.Thus, the type of flooring material greatly affects the total cost of these types of capacitive systems.By covering the floor with cheaper materials and using alternative construction for the floor electrodes, the total cost can be brought down to less than half of the implementation costs of the TUT Smart Home system.
The above compares well with the costs involved with other passive positioning system implementations.The capacitive positioning system of Rimminen et al., implemented with the UPM-Kymmene Oyj sensor laminate and a thin carpet covering the floor, costs only about 45 e/m 2 [33].In contrast, the pressure-sensitive floor of Gator Tech Smart House costs about 140 e/m 2 [7] and can provide about the same level of accuracy than our tile-based system.The ultrasound sensors of Nishida et al. [24] can measure human position with much better accuracy, but their system requires a large array of sensors installed close to each other in the ceiling and the total cost of such a system could be estimated to be over 200 e/m 2 .Conversely, the infrared sensors of Hauschildt et al. [4] could be significantly cheaper, as only one thermopile-array module would be required in each corner of a room.Indeed, if one module would cost about 100 e in large quantities [43], the covering of a 10 m 2 room would cost only 40 e/m 2 .Nevertheless, the reflection and dynamic background radiation effects of thermopile detectors still would need to be solved so that they could be used reliably in real living environments.

Safety
The methods presented here can be regarded as completely safe for human beings because the electric and magnetic fields in the human body are well below the levels set by the International Commission on Non-Ionizing Radiation Protection (ICNIRP) [5].The commission has set the limit for maximum internal electric field strength in the human body to 4.32 V/m at a 32 kHz frequency and the first harmonics of the square-wave signal which was used.Because the human body conducts these frequencies well, the potential difference between the feet and other parts of the body is only about one volt, as shown in publication [47].Hence, the internal electric field within the human body remains under 1 V/m, which is well below the recommended limit.
Because the wavelength of the 32 kHz square-wave signal used in the practical implementations and its harmonics are in the order of magnitude of kilometers, while the dimensions of the floor electrodes which were used are measured in meters, the emanated magnetic field from the transmitter is virtually nil.Thus, the human body's exposure to magnetic fields from the systems is negligible.Finally, the current levels in the human body need to stay below the ICNIRP set reference levels for general public exposure [5].In figures, the current in the neck was determined to be only 25 µA while the maximum current set by the ICNIRP for the 2.5-100 kHz frequency range is 6.4 mA.Thus, the current levels remain well under the set limit and do not pose any health hazards.

Future work
The test system of publication [51] installed in the TUT Smart Home can sometimes lose track of a person as already discussed in sections 5 and 9.1.In addition to the problem of the grounded objects electrically grounding the person in the kitchen area (see section 9.1), three other typical reasons for losing the track of the person were identified in publication [51].These were 1) the insufficient signal strength over a certain area of the apartment, 2) a person moving too fast and 3) a person moving in an unusual way, such as leaping on the floor, jumping from one non-conducting chair to another or crawling on the floor in areas of low signal strength.Although fixing all these problems is challenging, the first two problems could be fixed, or at least alleviated in many ways.First, a better receiver network could be created simply by placing more receivers in the environment.If the receivers could be placed, 133 Capacitive User Tracking Methods for Smart Environments for example, in the furniture, the contact with these items could also be recognized.Second, the structure of the floor tiles could be changed to yield a greater C t F .This could be done, for example, by moving the transmitting electrode closer to the surface of the tiles.In an installation in an old house this could be achieved by spreading the transmitting electrodes directly over the existing and non-conducting floor surface and by installing a new, thin floor surface directly on top of it.Also, the use of thinner carpets on the floor would help with this issue.Third, by decreasing the total number of transmitters, for example, by using only the larger 60×60-cm-sized electrodes, the scanning speed of the current implementation could be increased significantly.Fourth, new electronic circuitry that would enable faster scanning of the tiles could make a significant difference.Fifth, the tracking algorithm of the system could be enhanced significantly to help with the speed issue.Indeed, currently the transmitters under a person are not always scanned before that person moves to a new position.
The effects of shoes were not studied in detail in publications [47]- [51].However, they can have a significant effect on the ability of the system to track people and on the accuracy of the results [47,51].This happens, because C t F gets smaller in inverse proportion to the distance between the feet and the transmitters, and the insulation characteristics of shoes cause negative effects.Indeed, the measured C U can be only some tens of percent of its magnitude without shoes.Based on the short tests described in publication [47], the measured height of a person with shoes can vary between 50-80% from the measurements achieved when the person is not wearing shoes.Even though the correction factor k of Equation ( 3) partially quantifies the insulation characteristics of the shoes, other practical methods for quantifying the shoe insulation characteristics with tile-based positioning systems could be researched.However, if the transmitting electrodes were to be installed closer to the feet, as proposed above, this problem would also be alleviated.

Conclusion
This chapter studied capacitive user tracking methods for smart environments.The developed methods can be realized in an unobtrusive way to keep the living environment as a relaxing space.Thus, the people's lives in these environments can be supported and the inhabitants can concentrate on their primary goals rather than be burdened by the sensory actions of their living environment.Specifically, the chapter focused on the research work of Valtonen et al. and reported the used methods and key results of the developed and implemented tracking systems.Altogether, the methods presented in this chapter provide a reasonable level of accuracy for tracking people, and by using some of the developed sensing methods together, three-dimensional user tracking can be achieved in an affordable way.
The presented sensing methods could be used in many context and activity recognition applications for assisting the elderly at home.Indeed, by knowing the three dimensional human position and posture in a smart home, the home could reason the activity of the person with a much higher accuracy than without the position information.For example, the daily routines of the elderly could be monitored to support the independent living at home or to alert help if the elder should fall down and not get up.If family members have enough height difference, they can be recognized from each other and personal services could be provided.In addition, many applications in the areas of gaming and virtual reality could benefit from an unobtrusive and affordable human tracking system.

Figure 1 .
Figure 1.Capacitive sensing is typically implemented in one of its three primary sensing modes.The figure shows a) a basic measurement setup with simple measurement circuitry and the differences in b) transmit, c) shunt, and d) loading mode electrode and capacitance configurations.Although commercial capacitance measurement chips are widely available, simple measurement circuitry can be built consisting of only a signal source, a phase shifter and a synchronous detector.Modified from [42].

Figure 2 .
Figure2.A simplified capacitance model of the environment with a cutaway picture of the TUT Smart Home.The floor is constructed from large 60×60-cm raised floor tiles and the transmitting electrodes are attached to the underside of the tiles.The system's receivers are placed in the environment, for example, under the top of the table.[51]

Figure 3 .
Figure3.Two different types of tiles with different sizes of electrodes were used as transmitters in the implemented test systems of publications[48],[50], and[51].The left-hand side of the figure shows the bottom surface of a standard, commercial 60×60-cm raised floor tile from the TUT Smart Home.The inset photo shows a snap fastener soldered to the middle of the electrode for cable connection and a screw drilled next to it to prevent the snap fastener from being flattened.The right-hand side of the figure depicts a custom-built floor tile with four roughly 30×30-cm copper foil transmitting electrodes.These electrodes are partly hidden by the two plywood boards on the sides that stiffen the tile and raise it to the same height as the commercial raised floor tiles.[51]

Figure 4 .
Figure 4.The variation in the the capacitive coupling between the tracked person and the receivers is demonstrated in the above figure with a percentual received-signal-strength map.The map was obtained by walking slowly around the TUT Smart Home and by recording the largest measured C tU for each position.The highest signals were recorded around the dining table and are represented with the darkest contour lines, but the positions of the other receivers are also easily distinguishable.Receivers R1-R3 are built of conductive textiles and are either suspended from the ceiling (R1 and R2) or placed under the bedlinen (R3).R4 and R5 are constructed of copper foil that has been attached to the undersides of the living and dining room tables.[51]

Figure 5 .
Figure 5.An example of the gathered walking tracks during the long-term living test in the TUT Smart Home.The data was gathered between 20:00 and 21:00 hours on the third day of living in the TUT Smart Home.The activities during this hour are analyzed in section 6 and shown in Figure7.[51]

Figure 9 .
Figure 9.The response of the electric field ranging method was tested in publication[49] with two postures.First, a test user stood in the middle of the tracking area and swiveled through 360 • while keeping both feet on the ground.Second, he stepped forward in eight different directions while keeping the other foot in the same place.Modified from[49].

Figure 10 .Figure 11 .
Figure10.The electric field distribution around a 170-cm-tall human standing on a transmitting floor tile in a conductive building simulated with steel floor and ceiling.By measuring the capacitance between the body and the receiver above the person, both the human's height and posture can be determined.[47]Measurement setup A system for measuring a person's height can be constructed from a single transmitter and a receiver in transmit mode.The transmitter under the person's feet is used to emanate an electric field around the body and the C U is measured with a receiver

Table 1 .
[51]-based positioning results: the mean (µ) and standard deviation (σ) of the measured errors with the errors at different confidence levels.The maximum error in the table represents the 100% confidence level.[51] An example of one hour's worth of activity data acquired from the TUT Smart Home during a long-term living test.The one-minute-wide boxes are colored dark only if the test person touched the item during that one minute period.During this one hour, the test person 1) prepared food in the kitchen, 2) watched TV on the sofa or used a laptop on the sofa table, 3) ate food in the living room, 4)