1 Introduction

Nowadays in this world, technologies are advancing day by day. For this reason, maximum devices need to be updated with smart technology. The smart systems are organized by artificial intelligence (AI) and build smart equipment that makes the devices more interactive with the user. However, the smart device has the capability to easily sense, process and analyze the captured information. In modern days, working professionals are highly busy with their daily work and thus it is quite difficult for them to check daily necessary information including the latest news, To-do list, daily stock market update, social media newsfeed, traffic jam update, weather forecast and so on. This research proposed an IoT based smart mirror that helps the users to receive all this information. In addition, an AI-based face detection method also introduced that ensures a specific level of security for the proposed architecture. Besides, this system will also offer the opportunity for users to check their daily e-mails, helps them by playing preferred audio and video songs with the features of voice input. Currently, significant research works have been conducted in the domain of developing an interactive smart mirror for personal use. Rabben et al. [1] developed a smart mirror-based solution allowing pupils to wash their hands without the assistance of teachers or adults. This technique was created for the students in kindergartens and schools who require the assistance of a teacher to wash their hands in order to keep them hygienic. Despite the fact that this method gives feedback and instructions, it is quite popular among general pupils. Besides, by presenting the basic information of the users, Akshaya et al. [2] have developed a two-button supported smart mirror, one for the website and another for the map. The key feature of this research is to access the smart mirror from several areas with an assistance of a user-friendly interface. Purohit et al. [3] used facial recognition to construct a virtual assistance-based smart mirror solution for home automation. This suggested device may be utilized as a regular mirror as well as a smart mirror to obtain the user's daily information. The Raspberry Pi is employed to control and power the equipment in this system. However, in other work, D'souza et al. [4] have built a smart mirror that allows users to manage their home appliances using speech recognition approach. This application is primarily designed to reduce users' energy use while also displaying some essential functions such as time, traffic, Google Calendar, weather, cryptocurrency, and so on. In another recent work, Halaby et al. [5] presented a smart medical mirror that would display the user's vital signals such as SPO2, heart rate, BMI, and body temperature. Surprisingly, the proposed system is time-consuming, it takes more than 5 s for initializing, measuring and displaying the result. However, for those who are having difficulty getting medical check-ups due to laziness or busyness, this method may be able to aid them in obtaining their basic vital signs, which can help them spot deadly illnesses and medical problems at an early stage. Similarly, Nadaf et al. [6] and Njaka et al. [7] also developed a smart mirror for ensuring home security along with some basic features. A biometric authentication-based user recognition model has developed by the authors [7, 8] to ensure sufficient security in accessing the smart mirror. To detect an intruder through a smart mirror in a room, Jin et al. [8] have proposed an alarm-based interactive application. This article focuses on storing images of users in a dataset and the designed system sends notifications to the authenticated user when the model does not identify someone. With an 80% accuracy, in detecting user’s faces Mohamed et al. [9] have designed an artificial intelligence-based smart mirror. For recognizing existing users, they also proposed a registration method with the support of the facial recognition method. In recent times, Hollen et al. [10] have designed a facial recognition-based smart mirror to detect the user’s mood in real-time. The authors employed the concept of the user’s face and outbound movement detection to recognize the face and identify the mood in the system. In this article, the design and implementation of an IoT based smart mirror with the support of personalized information recommendation and face recognition approach is proposed. Following Table 1 highlights the available features for both categories of users (general and authenticated). This research has focused on the design and development of an interactive smart mirror; the development of a voice-controlled input; and to ensure the accurate detection of user’s faces to access the features.

Table 1 User wise features availability of MirrorME architecture
Full size table

2 Methodology

In this section, the design methodology of this proposed framework has been discussed. The proposed architecture is suitable for the essential functionality of any smart home. On the other hand, this will timely remind the user for a specific task based on the user's preference. Different features of the proposed MirrorME application are illustrated in Fig. 1. MirrorME is designed dedicatedly for two different categories of users; one is general users and other is authenticated users. The authentication process is developed with the support of HOG-SVM based face recognition approach. Regarding the access of different features, there are some personal information recommendation features for any existing users whereas the general users only have the access of some specific basic features. However, the authenticated users also access some of the premium features of the smart mirrors by using the voice recognition features. Functional and access diagram of general user and authenticated user of MirrorME application is shown in Fig. 2.

Fig. 1
figure 1

Various features of the proposed MirrorME application

Full size image
Fig. 2
figure 2

Functional and access diagram of general user and authenticated user of MirrorME application

Full size image

As we have already discussed, in this exploration, face recognition is required to detect the authentic user for accessing the smart mirror. Face recognition has been employed in this research using the architecture of histograms of oriented gradients (HOG) features and the linear support vector machine (SVM). The proposed feature extraction algorithm by Dalal et al. [11] known as Histograms of Oriented Gradients (HOG). In HOG, initially, the image is divided into small cells and then the gradient or edge direction histogram is collected in each cell unite, and finally, these histograms are combined to form a HOG descriptor. While doing feature extraction, HOG generally creates 3780 features of the image.

A gradient in the horizontal and vertical directions is obtained first to extract the feature in HOG. The template of horizontal direction is \(K= [-{1,0},1]\) and to filter image, its transposition can be used, that is why it is possible to calculate the horizontal and vertical gradients, easily. The expression Eqs. (1) and (2) is given as follows.

$${g}_{ x }=I\left(x+1,y\right)-I(x-1,y)$$
(1)
$${g}_{y} =I\left(x, y+1\right)-I (x,y-1)$$
(2)

Here, the direction gradient of x is represented by \({g}_{x}\) and the direction gradient of y is represented by \({g}_{y}\), the pixel value of \((x,y)\) is denoted by \(I(x,y)\). The gradient magnitude of \((x,y)\) is denoted by \(\Delta g (x,y)\) and calculated Eq. (3) by

$$\Delta g \left(x,y\right)=\sqrt{{{g}_{x} }^{2}+{{g}_{y}}^{2}}$$
(3)

And, the gradient direction \(\left( \theta \right)\) of \(\left(x,y\right)\) is calculated Eq. (4) by

$$\theta = {\text{arctan}}\left( {g_{y} /g_{x} } \right)$$
(4)

In machine learning, support vector machine (SVM) [12] used as a binary classifier. Classes separated with the largest gap between the support vectors. Here, the borderline instances in a class known as support vectors. In addition, we have extended the SVM with the support of kernel. Kernels transform data from input space to feature space. The kernel is a mathematical function that takes two arguments and returns the dot product of their value. Let two data points \({x}_{1}\) and \({x}_{2}\) mapping is denoted by \(\varphi\) then the kernel \(K\) will be Eq. (5).

$$K\left({x}_{1}, {x}_{2}\right)= \varphi \left({x}_{1}\right) T \varphi \left({x}_{2}\right)$$
(5)

When input space and feature space are equal then it is known as linear kernel and this linear kernel is used in linear SVM. Mathematically, the equation of linear kernel can be expressed as Eq. (6).

$$K\left({x}_{1}, {x}_{2}\right)= {x}_{1 }T {x}_{2 } \Rightarrow { \upvarphi }({\mathrm{x}}) ={\mathrm{ x}}$$
(6)

Due to its capability of faster application training therefore, it is highly efficient in high-dimensional data applications. The proposed workflow is consisted of detecting faces using HOG and linear SVM, computing embedding and comparing the vector to the database via a voting method. The basic flow of the HOG feature extraction algorithm and the face detection technique of the proposed application is illustrated in Fig. 3.

Fig. 3
figure 3

Workflow diagram of feature extraction algorithm and face detection procedure using histogram of oriented gradients (HOG) and support vector machine (SVM)

Full size image

3 Design and implementation

In this segment, the design and implementation approach has discussed with the architectural view of hardware and software. The hardware of the proposed framework consists of various components including two-way mirror, raspberry pi 3B+, camera, microphone, wood frame, display screen, speaker and some others. Following Fig. 4 depicts the IoT based hardware configuration diagram of the proposed MirrorME application.

Fig. 4
figure 4

IoT based hardware configuration diagram of the proposed MirrorME application

Full size image

3.1 Raspberry pi

The Raspberry pi [13] is a small and powerful computer that uses its own OS known as Raspberry pi OS and this operating system is Debian-based. Importantly, this device is the core component of our designed smart mirror.

3.2 Camera

For the purpose of face recognition, this work needed to use a camera and our proposed algorithm is incorporated with the camera to identify an existing user for accessing the MirrorME application.

3.3 Two-way mirror

Basically, a mirror is a smooth polished surface where the image is created by the reflection. In this work, a two-way mirror is used in which one side is transparent and the other side shows its reflection. The concept of using a two-way mirror is similar to using a usual mirror and that same mirror also acts as a functional object.

3.4 Display monitor

In order to present the basic information including date/time, weather update, alarm, news headlines, traffic update and so on, a display monitor is essential in this model. Therefore, in the development of the model, LG 14-inch monitor is connected to the Raspberry pi module using HDMA interfacing.

3.5 Microphone and speaker

To utilize the features of the voice-based activity, a microphone is needed in the designed model. A single Bluetooth speaker is also attached to the system to receive feedback. However, all the output results can be received through the speaker.

Based on the above discussion, the complete model of the architectural diagram and framework design approach of the proposed system shown in Fig. 5.

Fig. 5
figure 5

Architectural diagram and framework design approach of the proposed system

Full size image

To avoid unnecessary access to personal features for specific users, this model has also designed a multi-phase user recognition system. For recognizing the valid and existing users, HOG (histogram of oriented gradients) and linear SVM (support vector machine) based face recognition model have been employed. The detailed working procedure of the proposed system highlighted in Fig. 6. In addition, a stepwise description of the working procedure presented in Table 2.

Fig. 6
figure 6

A complete flowchart of MirrorME application working procedure with the features of different user interactions

Full size image
Table 2 Working procedure of MirrorME application based on user activity
Full size table

Following Fig. 7 illustrates both the software view and implantation process of MirrorME application. In Fig. 7A, all the available activity is properly labelled with appropriate features. Apart from designing the hardware architecture, this research also developed a dedicated software view for a better understanding of this application. On the other hand, in the implemented view Fig. 7B–D depicts the process of using home interface, road traffic update, and YouTube videos respectively.

Fig. 7
figure 7

Software view and implemented view of MirrorME application

Full size image

4 Results and discussion

This section has discussed testing results based on experiments. In order to test our designed model, we have conducted a result analysis based on the experience users observed during the assessment of this application. This research work has been carried out in the machine intelligence lab (MINTEL) of the Dhaka International University. Ten (10) participants have chosen to participate in this experiment based on previous experience in interacting with smart devices. All the participants were chosen aged between 22 and 25 years. In order to achieve higher accuracy in the task of face recognition, we have chosen a frame rate of 50 and a resolution of 480 × 360. The resolution 480 × 360 has been taken because a high-resolution frame takes a higher time to process and a low-resolution frame normally does not provide high accuracy. Among the Ten (10) percipients, there were Four (4) female participants (FP) and Six (6) male participants (MP) to interact with the MirrorME system. For each participant, exactly Ten (10) responses have been recorded for the features of face recognition and others. The success rate for each individual is shown in Table 3. However, during the testing phase, the face recognition feature shows ~ 100% accuracy for the fourth male participant (MP4). On the other hand, it shows an accuracy of ~ 60% for the first female participant (FP1).

Table 3 Analyzing the accuracy/success rate of face recognition feature based on the participations of (n = 10) individuals
Full size table

After the testing phase of face recognition, the same ten (10) participants again interacted with the mirror with the voice input. Each of the participants provides ten (10) voice input to interact with the individual features including YouTube, Alarm, Traffic, and daily schedule. Table 4 presents the testing results of the voice input. However, due to the network connectivity issue, sometimes the voice input took a longer time to function properly.

Table 4 Analyzing the accuracy/success rate of voice input feature based on the participations of (n = 10) individuals
Full size table

For each participant, we have recorded exactly (10 × 10) responses for both face recognition and voice input features. After the result analysis, we observed the average success rate of ~ 86.75% during the time of interaction with the MirrorME. Table 5 shows the average success rate for each individual based on the data of Tables 3 and 4. In addition, we also highlighted the three highest average success rates for the second male participant (~ 97%), fourth male participant (~ 92%) and third female participant (~ 90%).

Table 5 Average success rate of each individual participants for both the face recognition and voice input
Full size table
Table 6 A comparative features analysis of designed MirrorME application with others research work
Full size table

A comparative analysis with feature assessment matrix also designed and shown in Table 6 to compare the features of MirrorME application with other research work. This exploration observes that our suggested system outperforms other recent work in the categories of general information, reminders, social media alerts, voice recognition, voice control, face recognition, email notification, google map, weather update, and YouTube, based on a comparison with other recent work. However, several types of research in the field of voice control and face recognition have been fairly successful, for particular, the authors in [7, 8] integrated voice control and facial recognition to provide security for their recommended smart mirror. After analyzing the results, it is also discovered that combining speech and facial recognition provides a more secure system than other techniques. Similar characteristics of MirrorME have been proposed and achieved by the authors earlier in [9, 10, 16], however, the feature assessment matrix demonstrates that their produced application is still lacking in several areas, such as social media notifications, voice recognition, and voice control. Overall, the originality of this investigation is the design and development of an interactive smart mirror capable of conducting daily tasks while providing the highest level of security in terms of face and voice recognition for end users. Based on the feature assessment matrix, we observed that our proposed system contains every feature that considered as a complete interactive smart mirror and has outperformed other work in the specific category.

The research's next focus will be on the creation of a comprehensive home security gadget. The burglar may be spotted with this smart mirror, and the owner may be notified through email or text message with a real-time image of the intruder.

5 Conclusion

This paper demonstrated a smart mirror device with a user-friendly architecture with many impressive features. Following a service-oriented approach, a stable and easy-to-use architecture was also introduced in this article. However, security issues cannot be overlooked in today's world of interconnected devices. Therefore, this system was designed with a strong authentication framework to ensure the system's end-to-end security. Two different features (face recognition and voice input) set this designed model aside from other similar works. Furthermore, unique features such as information customization for each individual user made this model more convenient and effective. Again, the prototype's advancement has infinite possibilities in the future in the field of medical data analysis including BMI calculations, temperature check and blood pressure indicator. The most notable aspect of this model is the ability to carry the smart mirror display across the entire house.