Accessible Driving Game With Haptic Feedback + Sounds

This project is a wireless joystick-controlled LED matrix driving game designed for use with the Boston College Campus School. The Campus School at Boston College serves students ages 3 to 22 with extensive support needs, including complex medical needs, and brings together educators, therapists, nurses, undergraduate volunteers, and the BC community.

The goal of this project was to create an interactive, accessible, and exciting game that could be controlled with a large physical joystick instead of a keyboard, mouse, or small controller. The player moves a car through a pixel-art road scene on a HUB75 LED matrix and tries to collect 3 glowing orbs. When all 3 orbs are collected, the game plays a celebration sound, shows a cheerful message with fireworks, and then automatically restarts. (see attachments below for a brief video of gameplay and the celebration screen)

The project is split into two main units:

1. Display Unit: This unit contains the HUB75 LED matrix, Raspberry Pi Pico 2 W, speaker amplifier, and sound system. It displays the game, plays background music, plays orb and win sounds, and receives wireless joystick data.
2. Joystick Unit: This unit contains a second Raspberry Pi Pico 2 W, an analog joystick, a 3D-printed throttle handle, and a vibrating motor for haptic feedback. The joystick sends wireless control data to the display unit over Wi-Fi.

The final build uses laser-cut 1/8 inch birch wood, 3D-printed joystick handle parts, two mobile power banks, hot glue, foam padding, and simple electronics.

1.) 2x Pico 2w - https://learn.adafruit.com/quick-start-the-pico-w-with-wippersnapper

2.) 1x Speaker plus amp- https://www.adafruit.com/product/3351?gad_source=1&gad_campaignid=21079227318&gbraid=0AAAAADx9JvR7hzFouTtPGoDNIJzCnJQ3n&gclid=CjwKCAjwtcHPBhADEiwAWo3sJulwEmHZJepvcyJ_MvQTsWqjJ_pkGz9FWYvB3I7-psF1_TnrtKw8kxoCF90QAvD_BwE

https://learn.adafruit.com/stemma-audio-amp

3.) 2x Breadboard - https://learn.adafruit.com/breadboards-for-beginners

4.) 3x Vibrating motor - https://www.adafruit.com/product/1201

5.) 4 small quarter inch pieces of foam

6.) 2x 5000 mah powerbank

7.) hot glue + superglue as adhesives

8.) 1x hub75 display - https://www.adafruit.com/product/5036

9.) 1x Adaptor for display - https://www.adafruit.com/product/658

10.) 1/8th inch birch wood laser cut parts

11.) 1x 3d printed joystick handle (both right and left halves)

12.) 1x 1k ohm resistor

13.) 1x PNP transistor - IRF3205 - https://www.digikey.com/en/products/base-product/infineon-technologies/448/IRF3205/129405

14.) 1x Diode - 1N4007 - https://www.digikey.com/en/products/detail/diotec-semiconductor/1N4007/18833652

Other Part Quantities:

1. 1/8 inch birch wood laser-cut parts: As needed to cut the display and joystick enclosures
2. 3D-printed joystick throttle handle: see provided str files for 1 left half, 1 right half
3. Superglue (as needed to secure around the rubber joystick and connect the two halves
4. x4 1/4th inch pieces of scrap foam (for insulation and cushion inside joystick as needed
5. Hot glue (As needed)
6. Electrical tape (Used to secure electronics and wiring )

This project is a two-part wireless arcade game. The player controls a pixel-art car on an LED matrix using a separate handheld joystick unit.

The display unit shows the driving game on a HUB75 LED matrix. The player drives through the scene and collects glowing orbs. The goal is to collect 3 orbs. After the third orb is collected, the display plays a celebration animation, plays a win sound, and then automatically restarts the game.

The joystick unit is separate from the display and communicates wirelessly using the two Pico 2 W boards. The joystick button is not used in this version. Instead, the analog joystick movement controls the game, and a vibrating motor inside the joystick unit provides physical feedback.

You will need two Raspberry Pi Pico 2 W boards:

1. Display Pico: runs the HUB75 LED matrix game.
2. Joystick Pico: reads the joystick and sends wireless control data.

Install CircuitPython on both Pico 2 W boards. Once CircuitPython is installed, each board should appear on your computer as a drive named CIRCUITPY.

For each board, you will eventually copy its code file onto the board and rename it to: "code.py"

The display Pico and joystick Pico must be connected to the same Wi-Fi network.

Create a settings.toml file on both Pico boards with your Wi-Fi information:

The display unit is built around the HUB75 LED matrix and the Pico 2 W.

We used 1/8 inch birch wood for the laser-cut display enclosure. The display should sit securely in the front of the case, with room behind it for the Pico 2 W, HUB75 adapter, speaker amp, wiring, and power bank cable. *Note* we don't use the side covers in our final product for ease of access to internal components for troubleshooting. This approach also allows for better, less muffled sound since the speaker is mounted inside the display casing. As our end user will have the display placed for them by a teacher, and only interacts with the controller we decided this was a good end solution for us, but they are included in the file in case future makers want a fully enclosed cover for the display.

When assembling the display unit:

1. Place the HUB75 matrix into the front opening.
2. Mount the Pico 2 W and HUB75 adapter behind the display.
3. Leave access to the Pico USB port so the board can be powered and reprogrammed.
4. Leave space for the speaker and amplifier.
5. Use hot glue only after testing that the wiring and code work correctly.

For temporary testing, alligator clips or jumper wires are fine. For the final version, soldering or very secure jumper connections are better because the game may be moved, bumped, or handled during use.

Connect the HUB75 LED matrix to the display Pico using the HUB75 adapter.

The display code is written for a 64 by 32 HUB75 matrix. The Pico uses the HUB75 adapter pin mapping in the code, so the easiest method is to use the adapter rather than hand-wiring every matrix pin.

The display code uses the following HUB75 connections:

RGB pins: GP2, GP3, GP4, GP5, GP6, GP7

Address pins: GP10, GP16, GP22, GP26

Clock: GP11

Latch: GP12

Output Enable: GP13

Make sure the matrix is powered properly. The Pico should not be expected to power the LED matrix directly from a GPIO pin. Use the correct power input for the HUB75 matrix and make sure the Pico and display share a common ground.

The display unit includes sound effects and background audio.

The game code uses:

background.wav

ding.wav

win.wav

Place these audio files on the display Pico’s CIRCUITPY drive.

The code expects them at the root level:

For the speaker amp:

Amp power/red wire -> Pico VBUS or 5V power

Amp ground/black wire -> Pico GND

Amp signal/white wire -> Pico GP18

If your amplifier has an enable or shutdown pin, connect it to GP8

The current display code turns GP8 on as a speaker enable pin. If your amp only has three wires, power, ground, and signal, you can ignore the enable pin.

Test the speaker before gluing it into the case. Once verified, we used hot glue for this.

Download the attached file: hub75_cargame_pico-1.py

Copy it to the display Pico’s CIRCUITPY drive and rename it to "code.py"

Also copy the three sound files to the same drive:

1. background.wav
2. ding.wav
3. win.wav

When the display Pico starts, it connects to Wi-Fi and waits for joystick control data. Open the serial monitor to find the display Pico’s IP address. You will need this IP address for the joystick code.

The joystick unit holds the analog joystick, the joystick Pico, the vibration motor circuit, and the battery cable.

We used laser-cut 1/8 inch birch wood to create a sturdy joystick box. The joystick is mounted on top so that the player can easily push it forward and move the game.

The joystick button is not used in this version of the project. Only the analog joystick motion is used.

Before final gluing:

1. Place the joystick in the top opening.
2. Place the Pico 2 W inside the box.
3. Place the vibration motor where it can be felt through the case.
4. Use foam pieces to reduce rattling and support loose parts.
5. Test the joystick and motor before closing the box.

Depending on the solidity of your joystick unit, add rubber bands around the controller casing to spring the throttle back to the home position and reduce strain on the joystick unit when idle. This also increases stability of the overall unit.

Lastly, we add velcro to the bottom of the enclosure for easy mounting to a wheelchair tray or other accessible solution.

To make the joystick easier and more exciting to use, we added a 3D-printed throttle-style handle.

see attached files and print at their full scale

1. Throttle Left Half STL
2. Throttle Right Half STL

Print both halves of the throttle handle. After printing, place the two halves around the rubber joystick attachment on top of the joystick unit.

Use superglue to secure the two halves together around the joystick attachment.

Make sure the throttle handle is straight and comfortable before the glue fully dries.

The joystick unit uses the analog joystick as the main controller.

Use these connections:

1. Joystick VCC -> Pico 3V3
2. Joystick GND -> Pico GND
3. Joystick analog output -> Pico GP27

The joystick button is unused, so it does not need to be connected.

Use jumper wires or alligator clips while testing. For the final build, soldering is better if the joystick will be handled a lot.

The vibration motor should not be powered directly from a Pico GPIO pin. Instead, use the MOSFET as a switch.

Use the IRF3205 as a low-side MOSFET switch:

1. Motor positive wire -> 5V / VBUS
2. Motor negative wire -> MOSFET drain
3. MOSFET source -> GND
4. MOSFET gate -> Pico GP15 through 1k ohm resistor

Diode across motor:

1. Diode stripe/cathode -> motor positive side
2. Diode non-stripe/anode -> motor negative side

Also make sure the motor power ground and Pico ground are connected together.

The diode protects the circuit from voltage spikes created when the motor turns off. The 1k resistor helps protect the Pico pin going to the MOSFET gate.

Test the vibration motor before gluing it in place. Once working, hot glue the motors into the hollowed section of the joystick/throtle. Next, place the rubber attachment of the joystick unit into the bottom of the throttle unit. If needed, carefully trim away the excess rubber on the head of the joystick until it fits snugly into the 3d printed part. Close the top opening of the throttle unit wiht electrical tape or another creative solution. (there is also space to add a button to the top of the throttle if desired in a remake of this project).

Download the file: joystick_pico2w_sender-1.py

Copy it to the joystick Pico’s CIRCUITPY drive and rename it to code.py

Before running the joystick code, update the display IP address in the joystick code to match the IP address printed by the display Pico.

Look for this part of the joystick code:

DISPLAY_IP = "192.168.1.152"

Replace it with your display Pico’s actual IP address.

Both Pico boards must be on the same Wi-Fi network.

This project uses two separate power banks:

Display unit -> 5000 mAh power bank

Joystick unit -> 5000 mAh power bank

The display unit needs enough power for the Pico, LED matrix, and speaker. The joystick unit needs power for the Pico, joystick, and vibration motor.

Using separate power banks makes the project easier to move around and keeps the joystick fully wireless.

Power on the display unit first. Wait for the display Pico to connect to Wi-Fi and start the game.

Then power on the joystick unit. Once both boards are connected to the same Wi-Fi network, the joystick should begin controlling the game.

Test the following:

1. The LED matrix shows the driving game.
2. The joystick controls the car/game movement.
3. The background sound plays.
4. The orb collection sound plays when an orb is collected.
5. The vibration motor responds from the joystick unit.
6. The game ends after collecting 3 orbs.
7. The win sound plays.
8. The game automatically restarts after the win sequence.

Once everything works, secure the parts inside the cases.

Use hot glue to hold down:

1. Pico boards
2. Speaker amp
3. Speaker
4. Loose wires
5. Vibration motor
6. Power cables
7. Foam supports

Be careful not to cover the Pico reset button or USB port if you still need access to them.

Use foam pieces where needed to reduce rattling inside the joystick or display unit.

For final reliability, soldering is better than loose jumper wires. Alligator clips are useful for testing, but they can come loose during use.

1. Turn on the display unit.
2. Turn on the joystick unit.
3. Use the joystick throttle to control the game.
4. Drive through the pixel-art scene and collect the glowing orbs.
5. Collect 3 orbs to win.
6. After the win sound and celebration, the game automatically restarts.

The game is designed to be simple, colorful, physical, and easy to understand. The joystick handle makes the controller feel more like an arcade machine or driving simulator, while the vibration motor adds feedback when the player moves the joystick.

The joystick is not controlling the display

Check that both Pico boards are connected to the same Wi-Fi network. Then check that the joystick code has the correct display IP address.

The display works, but the joystick does nothing

Open the serial monitor on the joystick Pico and check whether it is sending data. Also make sure the joystick analog output is connected to GP27.

The speaker does not play sound

Check that the audio files are named exactly:

background.wav

ding.wav

win.wav

They should be placed directly on the display Pico’s CIRCUITPY drive, not inside a folder.

Also check:

Amp signal -> GP18

Amp power -> VBUS or 5V

Amp ground -> GND

The vibration motor does not work

Check the MOSFET wiring. The motor should not be connected directly to the Pico GPIO pin. Make sure the motor uses the MOSFET circuit and that the Pico ground and motor power ground are connected.

The game does not restart

The updated version should restart after the player collects 3 orbs. If it does not, make sure you are using the newest display code file.