Smart Fridge Helper

Fridge Helper is a smart fridge assistant built with a Raspberry Pi. It helps users keep track of food products before they expire or are forgotten in the fridge.

The user places a food package in front of a USB camera, captures an image, and scans it with a YOLO object detection model. The app shows the detected product type, confidence score, and lets the user save the product to a digital fridge inventory with an optional expiry date.

I made this project because food waste often starts with simply forgetting what is already in the fridge. Fridge Helper is a small prototype that combines AI, a Raspberry Pi, hardware feedback, and a web interface to make fridge tracking easier.

Main hardware:

1. Raspberry Pi 5
2. Razer Kiyo USB camera
3. I2C LCD display
4. RGB LED module
5. Freenove / Raspberry Pi project board
6. MicroSD card
7. USB-C power supply
8. Jumper wires
9. Breadboard
10. Resistors for the LDR voltage divider and button/LED testing
11. LDR / photoresistor
12. Push button
13. Multiplex wood for the laser-cut enclosure

Software and tools:

1. Python
2. Gradio
3. YOLO26n object detection model
4. FastAPI
5. PostgreSQL
6. Docker Compose
7. Google Colab / Jupyter Notebook for training
8. Roboflow / annotated dataset export
9. Inkscape / MakerCase for the enclosure design
10. GitHub for version control

A complete Bill of Materials with components, quantities, and prices is attached as a supporting file.

The first step was to set up the project environment and test the basic Raspberry Pi setup.

I started by checking that the main software parts worked together: the Gradio interface, camera input, model upload flow, database connection, and Raspberry Pi hardware communication.

At this stage the project was still very early, but it gave me a working base to build Fridge Helper on top of. I could run the Gradio interface from the Raspberry Pi and open it from my laptop using the Pi IP address.

The first frontend prototype used local JSON/mock data. The YOLO model and final database connection were planned for later.

I also made a rough system diagram. The idea already included a Raspberry Pi, camera, LDR/photoresistor, button, LCD display, RGB LED feedback, and possible buzzer feedback. Some parts changed later, but this diagram helped define the direction of the project.

After the first prototype setup, I started working on the main project structure. This included the dataset, database design, frontend wireframes, customer profile, and first model validation.

For the AI part, I chose YOLO object detection because Fridge Helper needs to detect food package areas in an image. I collected and annotated product package images in Roboflow.

The dataset focused on five classes:

1. milk carton or plastic cup
2. yogurt cup or lid
3. egg carton label
4. meat package sticker
5. cheese package sticker

I prepared the dataset for training by splitting it into train, validation, and test sets. The preparation script checked for missing labels, empty labels, invalid class IDs, and wrong bounding box values. This was important because bad labels would make the YOLO model harder to train.

Next, I made the first database ERD. The database was designed to store scan events, detected products, confidence scores, expiry dates, and inventory items. This helped me think about how the AI result would become useful data in the app.

At the same time, I created the first Figma wireframes for the required pages. These wireframes helped me plan how the user would scan a product, see the result, and save it to the fridge inventory.

I also wrote a customer profile and value proposition. The target users are students and families who often forget what food is already in the fridge. The main value of Fridge Helper is giving them a quick overview and reducing food waste.

Finally, I trained and validated the first YOLO26n model. The validation results included per-class metrics, a confusion matrix, and prediction examples. These results showed that the model could detect package labels, but also showed where the dataset and model could still improve.

After the software and data planning, I started working on the physical enclosure for Fridge Helper.

The goal was to make a small case that could hold the Raspberry Pi, USB camera, LCD display, RGB LED feedback, button, breadboard, and wiring.

I first generated a box layout and then edited it in Inkscape. I added cut-outs for the LCD display, camera area, buttons, cable access, and hardware feedback components.

I used 6mm mdf wood because it is strong enough for a prototype and suitable for laser cutting.

After the first wireframes and prototype, I rebuilt the frontend into a clearer Gradio interface for the final project flow.

The goal was to make the app understandable for a user.

The final interface includes five main pages:

1. Dashboard
2. Inventory
3. Scanning
4. Settings
5. About

The Dashboard gives an overview of the fridge status. The Inventory page shows saved products, expiry status, categories, quantity, and scanned/manual items. The Scanning page is used to capture a product image, run the AI model, review the result, and save it to the inventory.

The frontend communicates with the FastAPI backend for the fridge inventory. If the backend is available, products are saved in the PostgreSQL database. If the backend is offline, the app can still use a local fallback file for testing.

In this step I connected the main hardware components to the Raspberry Pi.

First, I connected the Razer Kiyo USB camera. This camera is used as the live camera source in the Gradio scanning page. The frontend shows a live preview, lets the user capture a frame, and then sends that image to the YOLO model for detection.

Next, I connected the I2C LCD display. The LCD gives short physical feedback during the scan flow. It shows messages such as camera ready, photo captured, scanning, bad lighting, no product found, low confidence, scan successful, and saved to fridge. Product names are shortened so they fit on the display.

I also connected an RGB LED module for quick visual feedback. The RGB LED uses three GPIO channels: red, green, and blue. In the project flow I use green for a successful scan, red for failed detection or low confidence, and yellow for warnings or problems. Yellow is made by turning red and green on together.

Finally, I tested the LDR/photoresistor. The goal of the LDR is to measure the light level before scanning, so the app can warn the user when the product is too dark.

After testing the separate hardware parts, I connected the sensors and modules to a breadboard.

The breadboard was connected to the Raspberry Pi using jumper wires.

On the breadboard I connected the LDR/photoresistor circuit with a resistor voltage divider(10K). This circuit is used as a light check before scanning. If the lighting is too dark, the app can warn the user instead of running a bad scan.

I also connected the RGB LED module(in mine resistors were already built-in) and other hardware wiring through the breadboard. The LCD display was connected to the Raspberry Pi through I2C, and the camera was connected through USB.

After the wiring worked outside the case, I started assembling the final physical prototype.

First, I glued the laser-cut wooden parts together to finish the box. Then I placed the Raspberry Pi, breadboard, camera, LCD display, RGB LED, LDR/photoresistor, and wiring inside the enclosure.

I checked that the camera had a clear view through the front opening and that the LCD display was visible from the outside. I also made sure the LED feedback could be seen and that the wiring still fit inside the box.

(Do not pay attention to my cable management at this point)

After assembling the hardware and frontend, I tested the full system.

First, I started the PostgreSQL database and FastAPI backend with Docker Compose. Then I started the Gradio frontend on the Raspberry Pi and opened it from my laptop using the Pi IP address.

In the Scanning page, I placed a food package in front of the camera and captured an image. Then I clicked the scan button. The YOLO model detected the package type and returned a confidence score.

During the scan flow, the LCD and RGB LED gave physical feedback. For example, the LCD showed scanning messages and the LED changed color depending on the result.

After a successful scan, I added an expiry date and saved the product. The product then appeared in the Inventory page, where I could see the detected type, category, quantity, added date, and expiry date.

Possible future improvements are:

1. automatically reading printed expiry dates from packages
2. improving the dataset with more product types
3. recognizing specific brands or product names
4. improving the enclosure and cable management
5. making the light sensor more precise
6. adding better notifications for products that expire soon