Fatigue Detection When Driving AI Model

Ever crashed your car because you were too tired?

Lucky for you I made a fatigue detection model that detects when you are getting tired while driving.

Overview:

It uses a OpenCV tool combined with mediapipe face mesh to collect the keypoints from a face. It then uses a random forest classifier to predict a certain class i defined.

Lastly it will get the predictions and use a hardcoded method to get the final tiredness score.

The build is made out of 3mm abs, there are 2 small boxes: one for the camera + sensors and one for the

raspberry pi, lcd and buttons.

Code:

Code is on github

How to use step by step:

1. Plug the raspberry pi in and surf to the ip displayed on the lcd
2. Use the automatic data collector and click the train the model button
3. Click start session and enjoy your ride!

Raspberry Pi 5: ~ €192

Logitech C270 HD webcam: €24.90

HC-SR04 Ultrasonic: €8.46

Buzzer: €1.20

Red LED: €2.42

2 buttons: €2.41

I2C LCD Display: €5.95

USB-A to USB-C cable: €9.99

Jumper wire kit (male to male, female to male & female to female): €7.49

Raspberry Pi 15W - 3A - USB-C Power Supply: €8.95

Resistors: €7.74

First I intigrated the camera, lcd, buzzer, ultrasonic sensor and LED.

I made the openCV tool with mediapipe face mesh to show a live "face mesh" on a person on the gradio page. Then I made a button connected to a method to start the automated data collection.

How it works is that the page will ask what face to make in 3 seconds the user then makes the face and the tool makes 60 snapshots of the keypoints and appends them in a json file under the corresponding class name.

The classes are prolongued_eye_closure, staring_behaviour, yawn, head_tilt_left, head_tilt_right, head_tilt_up and head_tilt_down.

From the collected data I made a jupyter notebook in which I upload the dataset and tried out some different models. The best one for me was a random forest with gridsearch to cycle though different hyperparameters.

When the model was done training I exported it to a joblib file and used on the live face mesh tool to predict the class live to the gradio.

To calculate the final fatigue score I use the predictions from the random forest model and make the predicted classes have a certain "weight" each. For example prologued_eye_closure will make the buzzer and Led go off very quickly while it may take a few yawns to give a warning.

Ultrasonic sensor:

The ultrasonic sensor is used to check if someone is behind the wheel. If not it will turn the system off to save power. Its possible to be turned off in the config page.

Buttons:

1. One button turns the system on/off
2. The other button cycles through classes and hardware to configure them. A short press will go to the next item. A long press toggles the state of the item it is on. The item currently selected is displayed on the lcd. These options are also configurable on the gradio page.

Data page:

In the data page you can see what classes are detected the most during the session.

Debugging:

The debugging page shows the output from the hardware aswell as buttons to manually trigger the led and buzzer.