LingoLando: a Screen Free Discovery of Nature

Hi! I'm currently a first year student in Creative Tech and AI at Howest University in Kortrijk, Belgium and this is my end of year project LingoLando!

LingoLando is an AI-Powered Interactive Language Game for Kids. It's designed to teach children from ages 4-7, the farm animals in English and Dutch.

I combine the Raspberry Pi with a Raspberry Pi camera module 3 and a custom YOLOv26n model I trained locally using my own images I annotated on Roboflow.

The LCD display shows the animal being asked and through the audio, you'll hear the animal as well in the chosen language. Through the Neopixel 12 LED ring light, it gives consistent light to the barn, then flashes green when the animal is correct and red when it's wrong.

The game challenges the child to place the physical toy animal on the correct tile, for instance a sheep = Green grass, Horse = Green grass or Yellow barn.

I built it inside a barn made out of Lego to keep it fun for children, customizable and environmentally friendly.

1. Lego, lots and lots of Lego
2. Bricks, tiles, roof tiles, any sized plates. (I had to buy extra bricks and coloured tiles for €64.78 but this depends on what Lego bricks you have yourself.)
3. Lego 48x48 stud baseplate - €14.99
4. Raspberry Pi 5 kit with power supply - €129.95
5. Farm animal toys you have laying around
6. Can also buy from Hema, Dreamland, Bol.com (Between €8.95 - €19.95)
7. Active cooler for Raspberry pi - €5.95
8. I2C LCD display - €8.95
9. Raspberry Pi camera module 3 wide - €34.48
10. Raspberry Pi camera cable 50mm - €3.50
11. NeoPixel RBG ring light (12 LED) - €9.44
12. RFID reader module - €9.90
13. Extra jumper wires - €5
14. Will need a soldering iron and tin to solder the wire to the NeoPixel ring light
15. Red cardboard (32cm x 38.5cm - but cut this after you've built your barn in case you decided to make it smaller/bigger) - €3.50

Note: The price on my Bill of materials includes the rough price estimate of the Lego I already own.

Start with building a solid wall around your Baseplate, 2 studs wide.

It's very important to remember to leave a hole for the power supply of the Raspberry Pi. I have made this roughly 6 studs wide and 2 studs high (see image). I placed this as central as possible to allow the camera to be centrally located. The camera will be above the pi through a chimney so central placement is important here.

You can use any colour of Lego you want with whatever variety of bricks you have. Just make sure you're interlocking the Lego bricks (overlapping the seams of the bricks so it's more secure). But I strongly recommend adding any technic bricks or bricks with holes you may have around the pi casing and chimney (in step 2) to help with the airflow of the Pi.

After layer 6, you want to line the entire top layer of bricks with smooth tiles, leaving place for some studded tiles on all corners in the middle of each stretch of the wall (2x6 stud plate is best here). This makes the 2nd half of your wall completely removable, meaning if you have any issues, you don't need to take down the entire wall. I strongly suggest adding a change of colour for the removable layer so you can easily see where you placed it.

Then proceed building the layers and repeat the previous step after layer 12.

You want your wall to be 20 layers high of normal bricks (not counting the thin layers for removability)

Make the very top layer with a mix of flat and studded tiles too so you can remove the roof later when necessary.

Firstly, make sure you've installed the Raspberry Pi active cooler already. Also make sure you can already log in to your Pi by connecting it to your laptop via the ethernet cable ang logging in on power shell.

For this step, I recommend having nothing plugged in to the pi but be aware of where the connections are and what you need. I made my pi wireless so I didn't need to bother with the ethernet cable. However, I did keep all the extra ports accessible by adding a grey window I had, on the right for the Power button and just creating a hole where the USB ports can be connected.

The Pi itself is raised slightly off the base plate to help with the airflow using small Lego plates risen with some bricks (see in photo). The casing is 13 x 8 studs using normal but 1 stud wide Lego pieces. Start building upwards allowing the spaces for the ports you need accessible like I explained above. The wall needs to be 18 bricks high + 3 thin bricks to be able to remove sections easily. Again, this is done by adding all smooth tiles along the surface but then adding studded pieces on the corners and on the sides. I recommend doing this because you'll need to be able to access your pi.

Now it's important to note that later I actually built an extra wall 2 x 8 along the power button (leaving the hole for the window) for the camera and an extra support for the ceiling (the orange section in the picture).

For the brown (mud) floor tiles, they are placed 4 studs away from the left wall and 10 studs away from the back wall. So the first 4x4 tile should be placed from the 5th stud from the left and 11th stud from the back. The blue tiles (water) should be directly opposite the brown (4 stud gap between them), also 4 studs away from the right wall and 10 studs away from the back wall.

The yellow and green barns are exactly the same width but are a little longer (18 x 16 studs) They should be in line with the brown and blue tiles as you can see in the image. but, there should be a gap 2 studs wide between the brown and yellow tile section, and between the blue and green tile sections.

Pi camera: The Raspberry Pi 5 has a smaller, 22-way (0.5mm-pitch) "mini" MIPI ports compared to the older Pi 4, a specialized 15-way-to-22-way adapter cable is needed to connect the Raspberry Pi Camera Module 3. Located by the audio jack, you'll see 2 flat connectors labelled CAM/DISP 0 & 1, gently lift the black lip up and place the smaller side of the cable inside with the pins facing away from the black lip, gently push it back down. Connect it in the same way to the camera but the black lip may be slightly different so be careful. (On my camera you need to unclip the sides before you lift it).

For the Neo-pixel ring light, I needed to solder the wiring directly to the holes. Make sure you have the ground wire, volt and input. You do not need the output wire here so it's not necessary to solder it.

(Compare pin numbers with the pin image)

The ring light needs to be connected on pin 2 (5v Power), pin 6 or 34 (Ground) and pin 38 (MOSI) directly on the Pi pins.

The LCD also had to be soldered to longer cables but it is possible to get longer wires (highly recommend). The LCD is connected to pins 4 (5v power), 3 (SDA) and 5 (SCL)

The RFID is connected to pins 1 (3.3v Power), 19 (MOSI), 9 or 20 (Ground), 21 (MISO), 23 (SCLK), 24 (CEO), 25 (RST)

I built a casing to hold the pi camera using Lego plates with clips to hold a small bar, the bar holds the pi in place. I've then used Lego technic bricks to connect it to the base. This makes it adjustable in 3 places, making it easier to find a nice camera angle to take the images of for your dataset. It needs to be placed at the back of the barn, on top of the chimney. Place it 3 studs away from the front edge (so place the mounts directly on stud 4). When you place the ceiling, you'll need to take the camera off of the mount to connect the wiring and bring the camera and wire through the chimney.

For this step I used all the big flat plates I had. You need it to be the same size as the floor plate (48x48 studs).

Lay the plates on the table and leave a hole big enough for the camera and mount. The hole itself is 14 x 12 studs, there should be 16 studs on the right of the hole and 18 studs on the left of the hole. There should be plates 2 studs wide running behind the hole.

You also need a hole for the NeoPixel ring light to go through. I placed this 6 studs away from the camera and the hole itself is 6x6 studs (It should be in the middle so 21 studs left and right of the hole).

Now the holes are there, start layering on top with more Lego plates to keep it together and structurally sound. Make sure everything is interlocking with each other (covering the as many seams as possible). It should be able to be held by itself, if there are pieces falling off or not clicking together, reinforce it. I only use 2 layers of Lego plates but, there were some plates by the holes that were harder to reinforce so I placed a couple extra 6x4 plates on the corners of the holes to keep it secure.

To place the ring light, I made a small structure of 4 pink Lego arches with 1 round Lego brick underneath it (to look a bit nicer) and 1 pink brick on top to keep the arches together, we'll use these in a moment.

You need to make sure your LCD, RFID, Ring Light and Camera are all connected properly now. The RFID needs to go through the hole on the left of the chimney base (where the USB ports are). We'll connect that in the next step.

Now bring the ceiling to the barn and gently pull your LCD, Ring light and camera up through the hole. Connect your camera in it's casing to the mount and place it 3 studs from the front (as explained in the previous step). The LCD screen needs to connect to the very front of the barn in the roof so ensure it reaches. Place the ring light in to the hole, place the archway bricks between the wires and connect it together. I used a zip tie to keep the wires together, above the pink Lego structure but do not do it tight! it just needs to be there loosely but can be moved and tightened when the project is finished.

RPi

1. gradio/ : A folder that created the Gradio library for temporary files and management of the dashboard
2. ai/ : Houses the logic for the machine learning pipeline
3. api/ : Contains the scripts that allow different parts of the application communicate back and forth
4. core/ : Application logic
5. data/ : Stores all the images and JSON file
6. hardware/ : Where are your hardware scripts are for the LCD, Ringlight and RFID reader
7. inputs/ : A directory for media inputs such as the images captured by the camera
8. models/ : This has the Yolo training model but also the class for the neopixel light
9. services/: Background scripts to manage the setup
10. ui/ holds the styling layouts for builing the gradio interface
11. app.py : the main execution script
12. config.py : a settings file where the variables like pins, database paths and I2C addresses are configured.
13. list_camera.py: A utility script used to setup the camera with an option of USB cameras
14. requirements.txt : A file with a list of all python libraries used during this project

For the complete open-source project code, you can visit my Public GitHub Repository Link Here. This repository includes the core Python files, the database orchestrations, and the Jupyter Notebook used to train the machine learning brain.

If you want to expand the dataset or use your own custom animal toys at home, you can access my complete image library, annotations, and training versions directly on my Roboflow. You can build straight on top of my dataset or use the system to gather your own data from scratch!

Gathering Data with Gradio

When the LingoLando application is up and running, the Gradio web interface features a built-in camera utility. Use this tool to test your framing and make sure your camera is aimed squarely at your habitat tiles. Any snapshots you take through the interface are saved directly to RPi/data/snapshots on your local storage.

Once you have captured your dataset (I took around 2,000 images!), you can download that folder and upload it directly to Roboflow for labelling and retraining.

Installation

To make deployment completely seamless, the LingoLando backend handles its database and API servers inside lightweight Docker containers. This means you don't need to manually configure a local PostgreSQL database server on your clean operating system.

Preparing the Pi

1. Flash a fresh copy of Raspberry Pi OS (64-bit) onto your micro-SD card using the official Raspberry Pi Imager.
2. Boot up the Pi, connect to your network, open a fresh terminal, and ensure your package ecosystem is completely up to date: sudo apt update && sudo apt upgrade -y

Install Docker dependencies

sudo apt install git docker.io docker-compose -y

sudo usermod -aG docker $USER

(Tip: You need to reboot your pi right after running these commands to apply them.)

Clone the repo

git clone https://github.com/SpraggSherry/LingoLando-Smart-Barn.git LingoLando

cd LingoLando

Build and try

cd RPi/api

cp .env.example .env

docker compose up -d --build

cd ~/LingoLando

python3 app.py

If you decide to retrain the model with your own dataset, follow this workflow to build and deploy your new AI brain:

Labeling in Roboflow

On Roboflow, annotate all of your images. I also annotated every single habitat floor tile because the AI model needs to detect if the animal is standing on the correct designated tile.

Tip: If you press Ctrl + R on your keyboard for every new image, Roboflow will automatically copy the bounding box annotations from the previous image! This means you don't need to manually redraw the floor tiles 2,000 times—you only need to delete the previous animal box and redraw it where the toy moved.

Dataset Splitting and Augmentation

When you are happy with your images, split your dataset into Train, Validate, and Test sets. Go to the Versions tab on the left sidebar and click New Version at the top.

To ensure the model can reliably detect the animals under different lighting and angles, add these augmentations:

1. Horizontal flip
2. Slight rotation (between -15° and +15°)
3. Brightness adjustments (between -15% and +15%)
4. Blur processing (up to 1.4px)

(Feel free to add other augmentations and experiment)

Local Training on a Laptop

Open your project's Laptop folder in VS Code and launch the yolo.ipynb Jupyter Notebook file.

1. If you are just adding images to my existing project base, you won't need to change the first code box. If you are creating a completely separate model from scratch, update the red text fields with your specific Roboflow API details.
2. Run the second cell to train the model locally using your laptop's GPU. Training takes time—it took my system 1 hour and 50 minutes. Once it's done, you can step through the remaining cells one by one to see evaluation matrices on how well your model performs.

Deployment Tip: If you trained your model locally on a powerful computer rather than directly on the slower Raspberry Pi hardware, you only need to harvest the final compiled best.pt weights file. Download that file, transfer it to your Pi, and replace the old weight file inside your project directory to bring your new smart barn brain live!

The database is all inside the code, feel free to edit your own version in the RPi/api folder. The init.sql has the created tables with prefilled information. Sign in to the database with the information given in api folder.

You'll need to add your RFID card to the pi on the database - check the init.sql I mentioned above, you can see that I have INSERT INTO child - then the info like name and the RFID tag I have, so just make sure you update this before you build your docker container.

In the image above you can see the database structure with all the table information.

The LCD screen needs to be central at the front of the barn. I left a hole from the first line of Roof bricks around the edges 8 studs wide. Place 1 layer of studded tiles and 1 layer of flat tiles. The green board of the LCD slots perfectly between the Lego tiles so gently place him there, with the LCD screen leaning on the flat tiles you just placed. Place some 2 2x2 bricks on top of each other, against the wire connections on on the left side of the LCD wire (connected to the ceiling), this will keep the LCD in place while you build the roof.

The front and back are built with red Lego bricks 2 studs wide, just like the walls of the barn. I use roof tiles to do the sides and go 1 stud in every layer.

Build 1 layer at a time, remembering to move the roof tiles, one stud inwards with every round. I added some reinforced bricks underneath sections of the roof just to keep it structurally sound. I did this by placing 4x4 Lego bricks on top of each other to the same height as the roof tile and clicking it underneath, leaving 1 line of studs out to place 2x1 bricks so it can support the next roof tile.

The roof should be 6 red bricks high with an extra layer of Lego plates running all along the top. Make sure the corners and the middle of the roof have some studs sticking out (like picture) to connect the roof board. Cut the red cardboard to the size of the remaining gap in the roof, for this project it will be the measurements listed in the supplies (32cm x 38.5cm). Place some double sided tape or glue dots on the plates you've left slightly sticking over the side. Then place the cardboard on top, covering the gap in the roof perfectly.

With the code given, activate app.py and test the interface.

If you want to change the layout to your own needs, go to RPi/ui/dashboard.py - this is where the Gradio interface is built and designed.

Don't forget to check you added your RFID tag to the init.sql file

You should be able to log in once your RFID badge is scanned, test out the animals and check it works. Make sure the model detects when it's right or wrong.

Then you have a fun, interactive game for your child!

Enjoy!