Multifaceted Driver Fatigue Prediction Using Human Face Morphology Processing Technology

In recent years, the popularity of the automobile sector is increasing all over the world. But unfortunately, there exist an exponential increase in vehicles crime. At the present time, most vehicles are controlled via using mechanical keys, security cards, and password/pattern. But, with the development of IOT technologies and many embedded mechanisms, the vehicle security systems are continuously improving. Many well-known biometrics-based identification and verification techniques existed. Fingerprints, facial features, and iris have been employed in various security applications. Face recognition is considered a good choice biometric technique. For vehicle security and alarm systems because it based on human face feature information and can work under different conditions. This paper presents a proposal for the development of a vehicle guard and alarm system using biometric authentication based on Facial recognition system. And also, extent the framework to predict the drowsiness based on eye features monitoring. One problem commonplace too many eyes monitoring strategies proposed to this point is their sensitivity to lighting fixtures situation exchange. We can implement the hybrid system for combining both face recognition and drowsiness detection. A security method to implement in the driving environment, it is called the Driver Face Recognition-Anti Theft System. It involves face enrollment of the driver as the initial stage followed by facial recognition. And to detect the faces using convolution neural network algorithm and detect the eye states using HAAR Cascade algorithm with improved accuracy. In case of abnormal behavior that is drivers’ eyes found to be closed means automatically alarm can be ringed and also send notification to neighbors. Experimental results show that in real time driving environments using Python Framework and MYSQL as Back end.


Introduction
Driver drowsiness detection is a critical component of modern vehicle safety systems, particularly as the issue of drowsy driving gains more attention due to its association with accidents and fatalities.This technology is designed to address the dangers of driver fatigue, which can lead to reduced alertness, slower reaction times, and impaired decision-making, all of which increase the risk of accidents on the road.The effectiveness of driver drowsiness detection systems is based on a multi-step process.First, various sensors, including infrared and visible light cameras, steering angle sensors, and even EEG headsets, are strategically placed within the vehicle to monitor the driver's actions and physiological state.These sensors continuously collect data on eye closure duration, blink rate, head pose, and other behavioral indicators, as well as brain activity.The heart of the system lies in machine learning algorithms and computer vision techniques.Supervised machine learning models, such as neural networks or support vector machines, are trained on large datasets containing labeled instances of drowsiness.These models learn to recognize patterns and correlations between sensor data and the drowsy state, enabling them to make predictions in real-time.In practice, these systems operate in real-time, analyzing the driver's behavior as they navigate the road.If the algorithms detect signs of drowsiness, such as frequent blinking, drooping eyelids, or erratic steering, they trigger alerts.These alerts are designed to grab the driver's attention and can take various forms, such as audible alarms, visual warnings on the dashboard, or physical sensations like seat vibrations.Some advanced systems can also communicate with the vehicle's control systems to suggest corrective actions, like adjusting the vehicle's speed or recommending a rest stop.The overarching goal of driver drowsiness detection is to prompt the driver to take immediate action to mitigate the risks associated with drowsy driving.This might involve pulling over to rest, taking a break, or simply becoming more alert to the potential danger they pose to themselves and others on the road.As these systems continue to evolve and improve, they hold great promise in reducing accidents caused by driver fatigue.Manufacturers are increasingly integrating drowsiness detection technology into modern vehicles, making it a vital component of road safety in an era when distracted and fatigued driving is an ongoing concern.Nevertheless, it's essential to address privacy and data security concerns to ensure that these systems are not only effective but also respectful of driver's rights and personal information.

Literature review
Wanghua deng, et.al,… [1] developed visual object tracking that estimates the target position in each frame of the image sequence, given the initial state of the target in the previous frame.Lucas and Kanade proposed that the tracking of the moving target can be realized using the pixel relationship between adjacent frames of the video sequence and displacement changes of the pixels.The face, an important part of the body, conveys a lot of information.When a driver is in a state of fatigue, the facial expressions, e.g., the frequency of blinking and yawning, are different from those in the normal state.In this paper, we propose a system called DriCare, which detects the drivers' fatigue status, such as yawning, blinking, and duration of eye closure, using video images, without equipping their bodies with devices.Owing to the shortcomings of previous algorithms, we introduce a new face-tracking algorithm to improve the tracking accuracy.Further, we designed a new detection method for facial regions based on 68 key points.Then we use these facial regions to evaluate the drivers' state.In this study, we propose a non-contact method called DriCare to detect the level of the driver's fatigue.Our method employs the use of only the vehicle-mounted camera, making it unnecessary for the driver to carry any on/in-body devices.Our design uses each frame image to analyse and detect the driver's state.Federico guede-fernández, et.al,… [2] advanced driver assistance systems is the trustworthy drowsiness detection.The most widespread automatic drowsiness detection methods may be divided into three main categories based on: driving behavior, visual and physiological features.Driving behavior-based methods analyze information about the car position inside the lane, speed, usage of the steering wheel, brakes and gear changes.The main weakness of this method is the variation in accuracy for particular characteristics of the vehicle and driving conditions.The aim of this work is to propose a method for drowsiness detection based on changes in the respiratory signal.The respiratory signal has been obtained using an inductive plethysmography belt and it has been processed in real-time in order to classify the driver's state of alertness as drowsy or awake.This paper assesses the ability of the proposed algorithm to warn the driver when the early fatigue symptoms appear, thus it may be a valuable safety system in car environments to alert to these episodes.On the other hand, the breathing process involves several muscles which act on both inhalation and exhalation.Breathing is performed primarily by the diaphragm, a large muscle that separates the thoracic cavity from the abdominal cavity.In fact, the contraction and relaxation of diaphragm produce volume changes in the thoracic and abdominal cavities.Thus, the respiratory signal can be obtained from the tracking of the displacements of diaphragm, abdominal and rib cage.Muhammad ramzan, et.al,… [3] discovered the state-of-the-art research in the drowsiness detection system.The systematic review provides details of behavioral, vehicular and physiological parametersbased drowsiness detection techniques.These techniques are elaborated in detail and their pros and cons are also discussed.The comparative analysis showed that none of these techniques provide full accuracy, but physiological parameters-based techniques give more accurate results than others.Their nonintrusiveness can be reduced using wireless sensors on the drivers' body, driving seat, seat cover, steering wheel, etc. Hybrid of these techniques such as physiological measures combined with vehicular or behavioural measures, helps in overcoming the problem associated with individual technique thus results in improved drowsiness detection results like the combination of ECG and EEG features achieves the high-performance results emphasizing the fact that combining the physiological signals improves the performance instead of using them alone.Therefore, previous transportation system is not enough to handle these hazards on roads.Thus, by embedding the automatic fatigue detection systems into vehicles, several deadly accidents can be prevented.The drowsiness detection system continuously analyses the drivers' attention level and alerts the driver before the arrival of any serious threat to road safety.Rateb jabbar, et.al,.. [4] described an improved drowsiness detection system based on CNN-based Machine Learning.The main objective is to render a system that is lightweight to be implemented in embedded systems while maintaining and achieving high performance.The system was able to detect facial landmarks from images captured on a mobile device and pass it to a CNN-based trained Deep Learning model to detect drowsy driving behaviour.The achievement here was the production of a deep learning model that is small in size but relatively high in accuracy.The model that is presented here has achieved an average of 83.33% of accuracy for all categories where the maximum size of the model did not exceed 75KB.This system can be integrated easily into dashboards in the next generation of cars to support advanced driver-assistance programs or even a mobile device to provide intervention when drivers are sleepy.There are limitations to this technology, such as obstructing the view of facial features by wearing sunglasses and bad lighting conditions.However, given the current state, there is still room for performance improvement and better facial feature detection even in bad lighting conditions.There has been a significant rise in the number of vehicles that are equipped with Android Auto or Apple Car nowadays.Most of the cars that are being introduced now have these components built-in.Such features are now readily available in lower-end cars too.Due to this, drowsiness detection systems can be easily developed around such built-in Android and iOS platforms.Embedded devices or mobile phones that can easily pair with these car dashboards can be used to enhance driver behaviour detection using simple camera setup and state of the art computer vision systems powered by Deep Learning.Chao zhang, et.al,.. [5] present a method for drowsiness detection based on simultaneous detection of yawn, blink and BVP in this paper.9-channel SOBI is proposed to provide the main frame of our method by simultaneously separating multiple physiological sources.Short-term energy and kurtosis are combined to automatically identify the sources from the multi-outputs.Drowsiness is finally determined by analyzing the separated yawn, blink, and BVP signals in parallel.In this work, the emphasis is on reporting the SOBI based fusion framework for drowsiness detection.A very simple strategy was used here to merge multiple physiological information.To increase the robustness and adaptability to real driving environment, method that are more sophisticated will be employed in our future work.Furthermore, a better performance may be achieved if combining the framework with other wearable wireless sensors such as wrist band to overcome the drastic performance degradation in some extreme circumstances.As another important part of our future work, a demo based on the proposed method is now under development.The system is implemented completely on a smart phone without using any other external hardware.The system can play an audible alert if a driver is detected as in the drowsy state.The usage of smart phone greatly reduces the complexity of video acquisition, store and transmission.However, since driver's driving environment is highly dynamic, most of the smart phonebased algorithms are very susceptible to the dynamics in driving environment because the mobile platform cannot afford the computation and storage requirement for very sophisticated algorithms.The existing algorithms in mobile devices need further improvement to identify the driver's driving state in a more robust way.Venkata rami reddychirra, et.al,.. [6] proposed work a new method is proposed for driver drowsiness detection based on eye state.This determines the state of the eye that is drowsy or non-drowsy and alert with an alarm when state of the eye is drowsy.Face and eye region are detected using Viola-Jones detection algorithm.Stacked deep convolution neural network is developed to extract features and used for learning phase.A SoftMax layer in CNN classifier is used to classify the driver as sleep or non-sleep.Convolutional neural network (CNN) is used in the proposed system for detection of driver drowsiness.Since a feature vector is needed for each drowsy image to compare with existing features in a database to detect either drowsy or not.Usually, CNNs requires fixed size images as input so pre-processing is required.The preprocessing includes extracting the key frames from video based on temporal changes and store in database.From these stored images, feature vectors are generated in convolution layers of CNN.These feature vectors are then used for the detecting the driver drowsiness.CNN have layers like convolutional layers, pooling (max, min and average) layers, ReLU layer and fully connected layer.Convolution layer is having kernels (filters) and each kernel having width, depth and height.This layer produces the feature maps as a result of calculating the scalar product between the kernels and local regions of image.CNN uses pooling layers (Max or Average) to minimize the size of the feature maps to speed up calculations.In this layer, input image is divided into different regions then operations are performed on each region.In Max Pooling, a maximum value is selected for each region and places it in the corresponding place in the output.

Existing methodologies
In recent years, driver drowsiness detection systems have advanced significantly, with many automakers and tech companies incorporating them into modern vehicles.These systems not only monitor driver behavior but also leverage machine learning algorithms and sophisticated data processing techniques for more accurate and timely detections.One notable aspect of these systems is their ability to adapt to individual driver behavior.By establishing baseline patterns of alertness for each driver, the system can provide more personalized alerts.For example, if a driver typically blinks slowly, the system may not trigger an alert unless it detects a significant deviation from their baseline behavior.Moreover, some advanced systems combine multiple sensors and data sources for a comprehensive assessment of driver fatigue.For instance, they may combine data from facial recognition, steering behavior, and physiological sensors to increase the accuracy of drowsiness detection.These systems are often integrated with other safety features, such as adaptive cruise control and lane-keeping assist, to create a more holistic approach to driver safety.In some cases, they can even communicate with the vehicle's infotainment system to suggest playing lively music or providing recommendations for rest stops when drowsiness is detected.While these advancements in driver drowsiness detection hold great promise for road safety, challenges remain.Ensuring the reliability and accuracy of these systems is paramount, as false alarms or missed detections can have serious consequences.Furthermore, addressing privacy concerns regarding the collection and use of driver data is an ongoing consideration in the development and deployment of these technologies.Overall, the continued development and integration of driver drowsiness detection systems represent a significant step forward in reducing accidents and fatalities caused by drowsy driving, making roads safer for everyone.Several existing systems for driver drowsiness detection rely on a combination of sensors and technologies to monitor a driver's alertness and intervene when signs of drowsiness are detected.These systems often incorporate cameras placed strategically within the vehicle to track the driver's eye movements, including blink rate and duration of eye closure.Facial recognition technology is employed to detect changes in facial expressions and head movements, such as a drooping head or repeated yawning, which can indicate drowsiness.Additionally, steering behavior is analyzed to identify sudden or erratic movements, such as drifting out of the lane, which may suggest the driver is becoming drowsy.In some cases, physiological sensors like electroencephalograms (EEGs) or electrooculograms (EOGs) are used to directly measure brain activity or eye movements.When any of these systems detect potential signs of drowsiness, they trigger alerts, which can take the form of audible alarms, visual warnings, or physical stimuli like seat vibrations.These systems aim to prompt the driver to take immediate action, such as taking a break or becoming more vigilant, thereby enhancing road safety by mitigating the risks associated with drowsy driving.

Proposed methodologies
Driver authentication and drowsiness detection are important aspects of driver safety.Convolutional Neural Network (CNN) is a deep learning algorithm that can be used to train models for these tasks.The next step is data pre-processing, where the collected dataset is pre-processed to ensure consistency in image size, resolution, and quality.Once the dataset is ready, the next step is to train a CNN on the dataset to recognize the facial features of authorized drivers.The CNN could be a simple architecture such as Sequential, or a more complex one if needed.During the authentication process, an image or video of the driver's face is captured using a camera installed in the car.This image is then passed through the trained CNN, and the output is compared to the facial features of the authorized driver.If the output is a match, then the driver is authenticated and allowed to drive the car.If the output does not match, the driver is denied access.In this proposed system, we can implement the system for detecting the faces using HAAR Cascade algorithm and also track the eyes states with improved accuracy.In case of abnormal behaviour that is drivers' eyes found to be closed as a corrective action alarm signal will be raised.The system enters into analysis stage after locating the driver's head and eyes properly in image captured through camera.This image is then pre-processed using various Image Processing techniques for drowsiness detection.After pre-processing, facial features are extracted in both (Wearing mask and no mask) states.Eye features are detected based on inter and intra class variants for all peoples.Finally provide alert system in the form of alarm, SMS and Email alert admin with face recognition.The suggested layout is shown in Figure 1.

. Interface creation
The advancement of technologies for averting drowsiness at the wheel is a key dilemma in the field of accident prevention systems.Preventing drowsiness during driving necessitates a scheme for precisely perceiving deterioration in driver's vigilance and a means for alerting and reviving the driver.Drowsy Driver Detection System has been developed, using a non-intrusive machine vision-based concept.This system offers a method for driver eye detection, which could be used for observing a driver's fatigue level while he/she is maneuvering a vehicle.In this module, we can capture the driver faces from real time camera.The driver face can be registered in admin interface.

Face recognition
The first step is data collection, where a large dataset of face images with varying poses, facial expressions, lighting conditions, and backgrounds is collected.The dataset should include images of individuals with diverse ethnicities, ages, and genders to ensure that the model can recognize faces from different groups.Once the dataset is ready, the next step is to train a CNN on the dataset to recognize facial features.During the recognition process, an image of a face is captured using a camera or uploaded to the system.This image is then passed through the trained CNN, and the output is compared to the facial features in the database of known faces.This module is used to enable the camera in system.Capture image from camera.User or driver enter into the system using key and recognize the person whether he is authorized or not using Convolutional neural network algorithm.If the user recognized means, provide the access to use the car.If the user not recognized means, block the user to access the car.

Face recognition
Each face detected is stored for half a second to crop the image in order to detect the eye in terms of wearing mask or glasses.Our proposed algorithm is used for eye detection.This algorithm divides the face horizontally into two segments i.e., upper segment and a lower segment.Upper segment contains the image between the forehead to the eyes, and lower segment contains the image between nose to the chin.We take into account the upper segment and lower segment is discarded.The upper segment again is divided horizontally into 2 segments, this time upper up segment from the forehead to an eyebrow and the upper lower segment from eyebrow to a lower eyelash.After the eyes have been extracted from the image it is then that the current frame is replaced by a new one.The eyes extracted are now categorized in two parts through vertical calibration -the left eye and the right eye.This module is to detect the vectors from overall face images.Separate eye vectors from overall face images.Calculate the eye lid opening conditions using HAAR Cascade algorithm

Abnormal prediction
After the eye has been detected, the next step is to detect the eyes condition either they are open or close, so for this purpose intensity values are used.A graph is plotted which calculates the intensity distance in the eye separately through the eye lashes and eye brow and check the state of an eye on this intensity distance.If distance is large, eye is close and when distance is less, eye is open.The distance can be evaluated by analyzing the samples of images.Both the eyes are binarized to determine the threshold value and then the results are produced.If the system encounters five consecutive frames with the eyes closed the alarm is triggered for the next five frames.

Notification system
In this module send notification to admin and also user at the time of abnormal prediction.If the eyes are closed less than 50% means, provide voice alert for self-assessment.Then send SMS alert using SMS gateway services.And also, abnormal face can be capture in email.

Algorithms 5.1. CNN Classification
The classification is the final step of the system.After analyzing the structure, each section individually evaluated for the probability of true positives.Eye states are classified using Convolutional neural network algorithm.CNNs represent feed-forward neural networks which encompass diverse combos of the convolutional layers, max pooling layers, and completely related layers and Take advantage of spatially neighborhood correlation by way of way of imposing a nearby connectivity pattern among neurons of adjacent layers.Convolutional layers alternate with max pooling layers mimicking the individual of complex and clean cells in mammalian seen cortex .A CNN includes one or extra pairs of Save parameters  of the CNN

Hard cascade algorithm
Facial features are extracted using Haar Cascade classifier which is based on the Haar Wavelet technique to analyse pixels in the image into squares by function.This uses "integral image" concepts to compute the "features" detected.Haar Cascades use the Ada-boost learning algorithm which selects a small number of important features from a large set to give an efficient result of classifiers then use cascading techniques to detect face in an image.The Features extractions can be shown in fig 2.

Figure 2: HAAR like features
Step 1: The image (that has been sent to the classifier) is divided into small parts (or sub windows as shown in the illustration) Step 2: We put N no of detectors in a cascading manner where each learns a combination of different types of features from images (e.g., line, edge, circle, square) that are passed through.Supposedly when the feature extraction is done each sub-part is assigned a confidence value.
Step 3: Images (or sub-images) with the highest confidence are detected as face and are sent to the accumulator while the rest are rejected.Thus, the cascade fetches the next frame/image if remaining and starts the process again.

Experimental results
In this paper, we can implement the framework in Python language.And visualize the concept in following figures.However, careful design and optimization are required to ensure high accuracy and efficiency in both face authentication and drowsiness detection.Overall, the use of these algorithms represents a promising approach to improving safety and security in various settings, such as transportation and public spaces.The combination of face authentication and drowsiness detection using CNN and Haar Cascade algorithms has the potential to revolutionize the transportation industry.By using real-time facial recognition and drowsiness detection systems, drivers can be quickly and accurately identified and monitored for signs of fatigue or drowsiness.This could help prevent accidents caused by driver fatigue, which is a significant cause of road accidents worldwide.Moreover, face authentication can also be used for identity verification in other settings, such as airports, banks, and government agencies.The accuracy and speed of the CNN algorithm make it an ideal solution for these applications, where security and convenience are critical.In phase1, we can develop the framework to authenticate the users who are entered into the car.

Figure 1 :
Figure 1: Proposed framework 4.1.Interface creationThe advancement of technologies for averting drowsiness at the wheel is a key dilemma in the field of accident prevention systems.Preventing drowsiness during driving necessitates a scheme for precisely perceiving deterioration in driver's vigilance and a means for alerting and reviving the driver.Drowsy Driver Detection System has been developed, using a non-intrusive machine vision-based concept.This system offers a method for driver eye detection, which could be used for observing a driver's fatigue level while he/she is maneuvering a vehicle.In this module, we can capture the driver faces from real time camera.The driver face can be registered in admin interface.

Figure 5 :
Figure 5: Drowsiness detection at the time of wearing the mask

Figure 6 :
Figure 6: Drowsiness detection when wearing glasses 7. Conclusion In conclusion, the use of CNN and Haar Cascade algorithms for face authentication and drowsiness detection can significantly improve driver safety and identity verification.The CNN algorithm can be trained on large datasets to accurately recognize faces, while the Haar Cascade algorithm can detect drowsiness based on patterns in the real time face data.Both algorithms have been shown to be effective in real-time applications and can be integrated into existing systems for added safety and convenience.