MIRROR – PART OF A SMART HOME

. This paper presents a possible implementation of a personal assistant and control interface for a smart home. A prototype is presented, featuring functions such as appliance and light control, a map system based on Google Maps and various informational data such as calendar entries and news. The device takes the form of a voice controlled mirror, allowing for integration in existing systems as a replacement for a common household item, without the need for additional space


Introduction
In an age where one can share what he just ate using his fridge, or accept calls using one's watch, it has become a challenge to find ordinary items which may yet be modernized [1].The current approach to smart home solutions is the addition of a central control panel, commonly a touch device, accompanied by a commercial handheld device with an appropriate app installed [2].
This paper presents an alternative approach, where a common household item -the mirror -is equipped with a control panel's functionalities without impeding its inherent use.This is done by creating a voice controlled interface, thus making the interaction appear more natural and eliminating the need for physical interaction and cleaning.Additionally features of a personal assistant are added, allowing the user to instantly get any required information by quickly glancing at a part of the mirror while getting ready, or specifically asking for it [3,5].

Hardware
The mirror is composed of a NTT Corrino 617 SU laptop running Ubuntu 14.04, with a IIyama LED 22" E2283 as external screen and an external microphone.Additionally, a 4 millimeter one-way mirror is put in front of the screen for the desired effect as seen in figure 1.The IIyama E2283 was chosen as screen due to its built-in speakers and connector layout -they are on the side, allowing for more compact cable management when connecting to the laptop, or any future processing unit.

Fig. 1. Reference drawing of the screen
Several ESP8266 microcontrollers have been added, connected to, depending on the intended use: DS18B20 digital thermometer for temperature sensors or a 230 V AC relay for controlling lights and other AC appliances.
The ESP8266 has been chosen due to its low price, power usage, processing power and most importantly Wi-Fi capabilities.For coordination of the above mentioned devices, a TP-Link 1043ND router running OpenWRT is used.

Software
The interface consists of a web page displayed by Google Chrome.This approach allows rapid development and sets a foundation for easy extensibility of functionalities.For voice recognition and speech synthesis capabilities, the Web Speech API is used.With its' specification published on 19 October 2012, it is one of the newer additions to web technologies [8].Although not a W3C standard, parts of the API are being incorporated into popular browsers.Interfacing with additional devices driven by the ESP8266 is done with the MQTT protocol -a machine-tomachine (M2M)/"Internet of Things" connectivity protocol [7].Due to its minimalistic nature it is well suited for use in microcontrollers.Information in the MQTT protocol is divided into topics, which any device can either subscribe or post to.For example, the topic /lights/kitchen/currentStatus could be subscribed to, in order to always be aware of the light's status, whereas the microcontroller responsible can post messages whenever the state is switched.
In order to provide personal assistant functionalities, a popular calendar application is used -Google Calendar API, which has been chosen mainly due to its free of cost nature.Nevertheless, interfacing with other, paid alternatives such as Apple iCloud is also possible.
Additionally, a Node.JS HTTP server is introduced to overcome the browser's sandboxed nature.Normally, HTML websites can be served locally using the file:// protocol.This approach could not be used for the following reasons: file:// is not seen as a secure connection by Chrome, and therefore, in accordance with the Web Speech API, prompts for user input every time a web page requests access to the microphone; the requests to the Calendar and Translate APIs could not be made due to Cross-site request forgery prevention measures; communication with MQTT is not possible using a Web browser alone.

Interface and implementation
The mirror provides some features by default, without the need for widgets or modifications.These include displaying the date and time, upcoming calendar events, local weather, post it functionality and maps.The interface itself is designed in an elegant, minimalistic manner, featuring soft fonts such as Helvetica Neue and light colors.Additionally, any of the base elements can be hidden by issuing the command hide (pl.schowaj) <component name>.A combination of these elements can be seen in figure 2.

Fig. 2. Upper part of mirror, depicts date, time, weather and calendar
The weather is provided using the simpleWeather jQuery plugin, and shown using the weathericons font.Any interaction with the Web Speech API is handled by the annyang!library.Any commands are easily added by issuing a command with a dict as argument, such as annyang.addCommands({'schowaj*term':voice.hide});,where the desired term is the key, and the function to be called is the value of the argument given, making maintenance and easy addition of commands possible.
The map functionality is provided using Geolocation and a custom styled map provided by the Google Maps.The user may ask the mirror for any location -the map in figure 3 was produced by asking: how to find to Mikolajczyka street in Opole (pl.jak dojadę do Mikołajczyka Opole) -with a correct result.Alternatively, the from location can be set in the settings, and the style of the map can be changed to the default one provided by Google Maps.The map can be dismissed by issuing the command: delete map (pl.usuń mapę).

Extensibility
Since not all possible hardware configurations and software functionalities can be predicted, a simple Widget API has been implemented to allow for custom extensions to be added [4].In order to add a new widget, these steps must be followed: The widget must be placed as its own directory inside the /js directory; its own directory must contain a file called script.js; it must be listed inside the config.jsfile as a plug-in to be loaded.
As an example, a widget for studying the Chinese language, specifically its characters, has been implemented, with a file structure as seen below: In order to ensure a proper loading order, so that no script is ran before its dependencies are available, the onload attribute is used.Later, the init function counts the amount of times it has been called, and once a certain threshold is reached, the actual initialization is ran.document.body.innerHTML+= canvasHtml; annyang.addCommands({"nowyznak":chinese.newChar});As seen in this example, widgets are able to easily do the following things: inject its' own dependencies, add HTML to the DOM and add new voice commands.

Interaction with hardware using ESP8266 and MQTT
The MQTT API has been designed to be as transparent as possible, as to facilitate further additions both in hardware and in software plug-ins.The MQTT Node.js library has been used [9], and three methods have been made available: /mqtt/subscribe?topic=name to subscribe to a topic, /mqtt/publish?topic=name&message=value to publish a message to a topic, and finally /mqtt/get?topic=name to receive the latest publication, along with the date received.All ESP8266 are written in Lua, under the NodeMCU environment.Additionally, the scripts are designed around the state machine as seen in figure 4.

Fig. 4. ESP8266 finite state machine
As an example, a lighting module using WS2812 has been implemented.The WS2812 is a RGB LED controlled by PWM, for which NodeMCU has a built-in driver starting with the 0.9.6 version (as of 14th June 2015 still in development).Since the WS2812 uses 5 V as its' native voltage, a logic level shifter has been used in order to drive it from the 3.3 V rated ESP8266.