1 Introduction

Electricity plays a pivotal role in modern life, offering the power needed to illuminate our homes and drive industries and economies. Its importance is emphasized by its contribution to lighting, heating, and providing energy for various appliances and devices that have become integral to our daily lives.

The increased number of devices dependent on electricity reflects the ever-growing integration of technology into our lives. Information Technology (IT) has a massive contribution to electricity consumption. From smartphones and laptops to households and the Internet of Things (IoT), such as refrigerators, air conditioners, and washing machines, as well as electric vehicles, the electricity demand has increased exponentially. These essential home appliances are crucial to our daily lives, contributing to the overall electricity consumption in households. Another example is data centers which exemplify a substantial electricity demand, necessitating the installation of thousands of storage devices to manage large datasets. Maintaining these devices at optimal functionality requires rigorous cooling at lower temperatures. As we become more dependent on technology, so does the importance of maintaining a robust, sustainable, and eco-friendly electricity infrastructure.

According to the latest data from [1], the worldwide net electricity consumption in 2022 reached 25,530 terawatt-hours. Several factors contribute to the high electricity consumption. One of them is the lack of awareness on how to reduce utility costs. For instance, leaving devices running for extended periods or neglecting to turn them off completely such as lights left on for days in offices or homes contribute to the problem. Surprisingly, this behavior persists even after individuals receive their electricity bills. Research conducted by the Ministry of Energy in the UAE; electricity consumption has shown a significant increase from 2003 to 2019 [2].

Another factor is that individuals frequently find it challenging to assess their daily electricity expenditures until they receive the monthly bill. Given that electricity costs are dependent on individual usage patterns, many users lack awareness of their consumption levels. This difficulty is amplified by the consistent usage of electrical appliances that draw power continuously.

Moreover, despite the presence of applications on platforms like iOS, and Google Play that aim to provide expenditure insights and reduction tips, a considerable number of users express dissatisfaction. Frequently cited criticisms include complex interfaces that make it harder for users to find solutions and include unnecessary requirements like inputting daily usage. Furthermore, certain apps overwhelm users with an excess of information that proves challenging to process. The overall lack of user-friendliness, coupled with a failure to underline the environmental consequences of excessive electricity consumption and ways to address it, adds to the overall challenge of effectively utilizing these applications.

This paper proposes a smart practical use for sustainable energy, specifically focusing on electricity. The proposed application assists users in forecasting their upcoming utility bills by utilizing artificial intelligence (AI) algorithms, taking into account current and past bills. Additionally, it employs a deep learning (DL) algorithm to identify electrical appliances and calculates the total electricity consumption based on the operating hours of each appliance. The rest of this paper is organized as follows: Sect. 2 discusses related work and previous studies, while Sect. 3 details the requirements and architecture of the suggested application. On the other hand, Sect. 4 covers its design and implementation. Section 5 presents the results and includes illustrative examples using the application. Finally, our conclusions and plans for future work are outlined in Sect. 6.

2 Related work

The global call for sustainable energy practices is driven by climate change and the finite nature of natural resources. Among these different forms of energy, electricity is included. Moreover, with the sophisticated capabilities of smartphone cameras and processing power, AR has found widespread application in various fields through mobile applications such as industry, healthcare, education, environmental sustainability [3], and many other fields. Likewise, in recent years, ML and DL have been widely employed and have found extensive application across various domains, including sustainability [4].

In the context of energy conservation, several studies have been conducted. In [5], an AR-based mobile application for tracking energy usage has been developed. However, this application is limited to enabling users to scan household appliances, obtaining details on the energy consumed by each device, and calculating the daily total energy usage. The application is simple and does not have any integrated AI capabilities. The study in [6] concentrates on the creation of an IoT-based energy meter, which is designed to oversee energy consumption by incorporating a GSM module. The primary aim is to transmit energy data to mobile phones via text messages. The authors proposed a solution to tackle the challenge where regular meter inspections are not feasible for accurately assessing energy utilization. The suggested remedy involved enabling remote monitoring of power consumption, thereby improving the precision of billing processes. Additionally, the system provides the capability to manage the load remotely, allowing for the toggling of power states (on/off) through message communication. Moreover, the implementation facilitates the transmission of SMS notifications to both the end-user and the authorized electricity board, serving as an update mechanism and enabling online bill payment.

A study done in [7] adopted the back-propagation neural network (BPNN) algorithm to forecast electricity consumption, considering various factors such as weather conditions, weekends, and holidays. The authors found out that integrating these elements, particularly weather and weekend/holiday factors enhances the prediction accuracy and decreases the mean square error. Another study on electricity consumption forecasting using Holt-Winters’ exponential smoothing method was introduced in [8]. The primary rationale for adopting the Holt-Winters exponential smoothing method for forecasting is that it works well with a small sample of data. The authors suggested a hybrid forecasting model named FOA-MHW, where the fruit fly optimization algorithm selects smoothing parameters for the selected method. In the context of the growing significance placed on sustainability, the authors in [9] incorporated gamification and persuasive design principles to provide a system in five design cycles that monitors employees' electricity usage on their computer-related equipment at work place.

The study in [10] addressed the challenge of low prediction accuracy in building energy management systems. The authors proposed the development of a predictive model using ML algorithms and the study was applied in real world context in Malaysia. The three ML algorithms considered in that study were Support Vector Machine (SVM), Artificial Neural Network (ANN), and k-Nearest Neighbor (KNN), and they were evaluated based on the metrics: root mean square error RMSE, normalized NRMSE, and Mean Absolute Percentage Error (MAPE). The results revealed that SVM consistently outperforms both ANN and KNN in the majority of cases considered in this analysis, demonstrating low RMSE, normalized NRMSE, and MAPE. The authors in [11] proposed predictive models for energy consumption in manufacturing systems, focusing on energy efficiency in various sectors such as electrical, metal, plastic, and food manufacturing in the USA. They employed ML algorithms to develop a predictive model for energy consumption, utilizing a dataset from the Department of Energy Industrial Assessments Centers (IACs). The study’s outcomes revealed that the Random Forest Regressor and DL produced the most reliable and accurate results. In [12], the authors aimed to identify the optimal method for predicting electricity demand in specific scenarios. They applied various ML algorithms to forecast electricity consumption, with the KNN model demonstrating superior performance, achieving an accuracy rate of 90.92% in predicting agricultural production.

Other existing applications in this domain include Emporia Energy [13], a smart home energy management system that displays electricity consumption, provides tips on saving electricity, and facilitates the visualization of corresponding bills. Energy Cost [2] is an application enabling users to create a registry of electrical appliances and estimate the cost of their electricity consumption. Energy Conservation [14] is a straightforward application featuring various energy-saving wiki articles, offering advice on sustainable energy practices. The Electricity Cost Calculator [15] is an online application that serves as a tool, allowing users to compute electricity bills for various appliances by entering parameters such as power in watts, quantity, duration, electricity cost per unit, and billing duration. Etihad Water and Electricity [2] is a simple application that enables users to view and pay their bills, though it lacks features specifically addressing sustainability concerns. Meters Reading [16] is another application that calculates water, electricity, and gas readings based on images taken by the user's mobile device or iPad, utilizing OCR dependencies.

The authors in [17] conducted a comparative study to predict appliance energy consumption in low-energy buildings. They considered Tree Model, Ensemble Model, Support Vector Regression, Linear and Statistical Regression Models, and Neural Networking (NN) algorithms implemented in Matlab. The author incorporated important parameters such as temperature, humidity, and wind speed into the analysis, as these factors have an impact on energy consumption. Accuracy metrics, including Root Mean Square Error, Coefficient of Determination, and Mean Absolute Error, were used to evaluate the performance of the models. The results of the study revealed that the NN model exhibited higher accuracy in predicting energy consumption. However, it is noteworthy that the authors did not specify the types of appliances considered in the study, which is crucial information for drawing conclusive findings. The authors in [18] focus was on developing a smart energy management system for a microgrid, which is like a small-scale power system. The system was using deep neural networks (DNN) to efficiently control power and predict energy demand considering factors such as how many people are around, their habits, income, appliances, and the weather. They evaluated the proposed model to other models like gradient boost and linear regression using metrics like root mean square error (RMSE) and coefficient of determination (R2), the results showed that their proposed deep neural network model was more accurate in predicting energy consumption patterns. The authors in [19] conducted a thorough examination of the current ML approaches employed to predict energy consumption but the focus was on the manufacturing industry. Similarly, the authors in [20] conducted research with a Taiwanese semiconductor corporation, utilizing data analytics to create energy efficiency models. The aim was to enhance manufacturing efficiency, reduce overall energy consumption, and optimize machine configurations in the semiconductor industry.

Extensive efforts and research have prompted exploration into energy management systems and smart grid technologies, aiming to enhance the efficiency of residential electricity consumption [21]. Although they offer several advantages, including real-time monitoring, reduced energy utilities, and increased sustainability, they are not affordable solutions, as not all countries and individuals can afford them. Additionally, other concerns such as privacy and security arise.

In the various studies mentioned earlier, some focus on developing predictive models for energy usage in industrial and manufacturing settings, while others aim for simplicity by raising awareness. Additionally, some studies utilize proposed predictive models specifically for home appliances. Our proposed solution presents a user-friendly application, designed for easy use. This application integrates diverse technologies like AR and YOLO object detection, providing forecasts for the electricity consumption of home electrical appliances for 3 months ahead. It notifies users when the electricity consumption has exceeded the regular monthly consumption rate, and offers detailed statistics on the individual contribution of each appliance to the overall consumption.

3 System requirements and architecture

3.1 Requirements gathering

We conducted a quantitative survey using a stratified random sampling method via Google Forms, targeting all families in the UAE. This population was divided to include three income-based categories: low, middle, and high income. The sample size was 100 participants. The survey consisted of 14 questions—13 close-ended and one open-ended. Among the respondents, 24% reported living in small apartments with two or three bedrooms. Additionally, 36% revealed that their electricity consumption utility fell within the range of US$150 to $300, which is considered somewhat high. From the survey responses, we identified important features for our sustainability-focused mobile application, including monthly electricity consumption tracking, cost limit notifications, energy-saving tips, appliance-specific electricity consumption insights, and quick reminders. The application aims to assist users in sustaining and staying within their budgeted electricity consumption.

3.2 System architecture

The system’s primary goal is to offer solutions for optimizing electricity consumption in household appliances. The application analyzes monthly electricity usage and suggests sustainable alternatives or assistance based on user input values. Furthermore, it forecasts future consumption and utility bills using data analytics techniques and ML algorithms. The application includes an AR feature, allowing users to target electrical appliances and view real-time electrical consumption indicators. Another important feature is a Chatbot that is integrated to assist users with frequently asked questions, utilizing a Natural Language Processing (NLP) artificial intelligence algorithm. The context diagram of the system is illustrated in Fig. 1.

Fig. 1
figure 1

The proposed application system architecture

Full size image

In the user interaction sequence, as shown in Fig. 1, consumers interact with the application by first registering electrical appliances in the system and recording the consumption rate of each of these devices. Subsequently, the consumer has the option to request information on electricity consumption, including predictions generated through the ML algorithm and appliance consumption rates determined by object detection. Additionally, consumers have the option to interact with an intelligent chatbot for assistance. The application facilitates the utilization of AR technology to scan home appliances, incorporating the state-of-the-art YOLO (You Only Look Once) DL algorithm for object detection. This integration allows the user to easily scan and identify home appliances providing accurate and real-time measurements of electricity consumption. By employing Yolo as an efficient algorithm for object detection, the application empowers the accurate identification of different home appliances. These requirements collectively address the needs of stakeholders and users, aiming to optimize electricity consumption while enhancing user engagement and system functionality.

4 Design and implementation

The following explains the design and the implementation of the proposed application.

4.1 Application design

4.1.1 Electricity consumption prediction design using ML

Since this is a mobile application using a smartphone, we opted for a linear regression algorithm to construct the electricity consumption forecasting model. In terms of CPU computing and power efficiency, linear regression is often considered one of the most computationally lightweight ML algorithms. Linear regression involves simple mathematical operations, which are computationally efficient and generally require lower processing power compared to other ML algorithms. The model was trained using the dataset found in [22]. The dataset captures the electricity consumption of electrical appliances. It includes features such as the number of hours each electrical appliance like air-conditioners, refrigerators, televisions, mobile devices, and washing machines operates per month. Finally, using this model, the feature allows the consumer to scan their bills each month, which help the model to predict future monthly bills.

As shown in Fig. 2, the electricity consumption for each month is predicted based on the previous bills entered by the consumer.

Fig. 2
figure 2

Electricity Consumption Prediction Design Using ML

Full size image

4.2 Object detection design using AR

To build the model, the YOLO (You Only Look Once) algorithm, which uses a Convolutional Neural Network (CNN) for real-time object detection along with AR technology displays the electricity consumption rate and based on the number of hours, it computes the total electrical consumption rate. To reach higher detection accuracy of objects, a large number of pictures for various appliances to train the model. The electrical appliance detected will be visually represented within a green or red frame based on the total electrical consumption rate, whether low or high respectively.

4.3 AR and ML implementation

Our proposed application integrates both AR and ML components in order to provide real-time detection of electrical appliances inside the house and accurate electricity consumption rates. This integration consists of several steps and interactions as listed below:

  1. 1.

    AR Object Detection This feature requires the use of the mobile camera to scan and detect electrical appliances. Once the user chooses this option, the mobile camera opens, allowing the user to view the surrounding environment and target any electrical appliance. The integrated YOLO algorithm activates the live camera and starts detecting objects (electrical appliances) with high accuracy since it is based on the CNN algorithm. As long as the camera captures the frame, YOLO will analyze each frame to determine the electrical appliances within the surrounding environment. Once an object is detected, AR technology adds an overlay visual representation as a boundary box or a frame around the detected object (electrical appliance) within the camera view, resulting in real-time feedback to the user.

  2. 2.

    ML Electricity Consumption Calculation The ML feature computes the electricity consumption based on the detected appliances and their usage amounts. Each electrical appliance has a known predetermined power consumption rate (wattage). Users input the number of hours each appliance was powered on, and then the proposed application computes the total electricity consumption rate for any detected object. The computation is done by multiplying the wattage of each electrical appliance by the number of hours it was used. The resulting consumption output is then reflected and displayed to the user in real-time through different colored frames (green and red) around the detected object. The green frame indicates the consumption rate is within range (low consumption), while the red frame indicates that the consumption rate is above range (high consumption) and the user must take proactive measures to save electricity.

  3. 3.

    Interaction between AR and ML Once the AR detects electrical appliances throughout the camera view, it visually displays each appliance surrounded by frames or an overlay visual representation like a boundary box of different colors (green and red) as real-time feedback to the user regarding the consumption rate. The ML model immediately processes the data associated with these detected objects to calculate the electricity consumption rate.

  4. 4.

    ML Prediction Model The linear regression model is utilized to predict future bills of electricity consumption. As discussed earlier, this model was chosen for its computational efficiency, making it suitable for implementation on mobile devices. We trained the model using a dataset that captures different features related to electricity consumption, such as the name of the electrical appliance and the number of hours each appliance operates per month. Features such as air conditioners, refrigerators, televisions, mobile devices, and washing machines are included in the dataset to effectively train the model. Once trained, the model can predict future monthly electricity consumption bills. Our current model is designed to predict bills for 3 months ahead. Users can view these predictions alongside other relevant information, such as current electricity usage and tips for optimizing consumption.

  5. 5.

    User Interaction Users interact with our proposed application through various options and functions from the main menu, such as initiating object detection or predicting electricity consumption. Both AR and ML models work simultaneously in response to user interaction to provide real-time feedback to the user regarding their electricity usage and consumption rate in a user-friendly and easy-to-use manner.

4.4 Application implementation

4.4.1 Development environment

To train our object detection model for the features, we used Darknet, an open-source neural network framework written in C and CUDA. The YOLO object detection algorithm will be employed in conjunction with Darknet, and using the OpenCV API. Furthermore, in developing our application, we employed two development environments. Android Studio was utilized to craft the application interfaces, ensuring a user-friendly and visually appealing design. On the other hand, PyCharm was employed to deploy the ML algorithms to integrate the YOLO object detection algorithm and the Darknet framework.

4.5 Illustrative example

Once the user launches the application, the user will be directed to the registration screen as shown in Fig. 4a. If the user already registered with the application, he will just sign in. Otherwise, the user creates an account. Once signed in, the user will get another screen to interact with the application to be able to either (1) start detecting objects (appliances), (2) predict the electricity consumption for the upcoming 3 months, (3) get tips on conserving electricity, and (4) chat with the chatbot. All these options are shown in Figs. 3, 4 and 5.

Fig. 3
figure 3

EnergySense application menu options

Full size image
Fig. 4
figure 4

Registration screen: a registration screen, b object detection, c computation of total electricity consumption rate, d statistics of electricity consumption per appliance

Full size image
Fig. 5
figure 5

Prediction of electricity consumption

Full size image

Once the consumer selects the option “Start detecting objects”, it allows the consumer to scan any home appliances using a smartphone camera. The latter detects those appliances then, the consumer will be prompted to input the number of hours during which the appliances were powered on. Then, the total electricity consumption rate for these appliances will be displayed, Fig. 4b, c.

The total electricity consumption is computed as the product of the wattage of the appliance and the number of hours out by the consumer. In the current implementation of the proposed application, seven appliances can be detected, which are: TVs, laptops, smartphones, microwaves, ovens, toasters, and hair dryers. Moving to the camera activity screen we can see that the application immediately starts detecting every electrical appliance in our detection list. When an appliance is detected, here the AR is applied and shows a frame around the appliance along with the type of this appliance, and the electricity consumption of that appliance.

There are two possible colors for the frame displayed around the appliance, green and red. The green frame indicates that the consumer did not exceed the regular monthly consumption rate. Each appliance recommends a regular rate of consumption per month [23]. Similar appliances might have different wattage values, but as mentioned earlier, this feature is based on estimation; hence, the application will store an average wattage for each appliance to allow the feature to calculate the daily electricity consumption. To give the consumer more customization, which would help get more accurate estimations, an extra option will be added to allow the consumer to enter the exact wattage value that is registered on the back of the appliance. After that, the electricity consumption of that appliance will show up on the screen [24].

At the end of the month, and after the consumer has received the monthly bill, they will be able to see how much of that bill their appliances consumed. A bar chart option, as shown in Fig. 4d, can be chosen to show the statistics of each appliance. The feature will continue to send alerts to the consumer for appliances that have been heavily used and have exceeded the normal consumption. The alerts will remind the consumer to follow the tips that help optimize the use of those appliances [9].

The option “Predict the electricity consumption”, Fig. 5, enables the consumer to input their past monthly electricity bills and the application runs the ML algorithm to predict the electricity bill for the upcoming 3 months as depicted by the figure below. The figure to the left shows the inputs of the past monthly electricity bills. The application shows an interactive screen, shown by the figure in the middle, displaying the amount of the bills as a graph. When the consumer clicks the graph, it shows the interactive information with the month and the bill value. The predicted bills are shown as an interactive graph in the Figure to the right.

5 Discussion

Our proposed application is designed to be prominent among other existing ones. What makes it unlike others is the features that it encompasses AR and ML algorithms to empower real-time electrical appliance detection and electricity consumption rate. In addition, it highlights the prediction capability of the electricity bills for the upcoming months. The proposed application with its features compared to other existing ones is summarized in Table 1.

Table 1 Feature comparison: sustainable EnergySense vs. others
Full size table

The proposed application has a great impact on the economy, society, and environment. It assists consumers in optimizing electricity usage and saving costs. Also, it aims to address the environmental challenges caused by ever-increasing electricity consumption. Moreover, by providing users with practical tips for optimizing electricity consumption habits, the application encourages a more sustainable approach to electricity use, aligning with global environmental conservation efforts.

In testing our proposed application, we conducted thorough evaluations on various appliances, utilizing past utility bills as a benchmark. The results were encouraging, demonstrating the application's ability to provide accurate insights into energy consumption patterns. Moving forward, we plan to expand our testing scope to encompass a wider range of appliances and scenarios, ensuring the reliability and the enhancement of our proposed solution.

Looking forward, our future plans for the proposed solution include establishing a collaborative relationship with utility companies, seeking to promote the widespread adoption of our framework and encourage sustainable energy practices.

6 Conclusion and future work

As the years progress, technological advances continue to make our lives more convenient. Throughout this research, we have provided a detailed implementation of an application focusing on the electricity consumption of household appliances. The primary objectives were to design and develop a mobile application empowering user with information on the cost of each electrical appliance and ways to reduce it. By optimizing the usage of household appliances through machine and DL algorithms, particularly utilizing object detection for detailed insights, the application seeks to save costs and contribute to environmental sustainability. The prediction functionality based on user-provided data not only enhances efficiency but also aids in long-term electricity planning.

In future work, we plan to include more household appliances in the application and to give more flexibility to the user to get a prediction of the electricity utility cost for any number of months instead of only 3 months. Additionally, we aim to consider other ML algorithms to test the prediction feature.