SpaceDrums - Play Drums in the Air

As a kid, I was into stuff that made zero sense to normal people. While other kids wanted action figures, I was asking my dad for biometric security systems. Since my parents weren't going to fund my mad scientist projects, I developed a survival skill: I just built things myself. And this habit stayed with me as I grew into the adult I am today.

When I saw a video of air drums online, the kid inside me screamed, "You have to make one of those!". Naturally, I didn't think twice before deciding to build my own pair. I only start big projects because I severely underestimate how hard they'll be, and I had no clue how many headaches were waiting for me.

Three months of coding, PCB design, and latency debugging later, I finally built Space Drums: an open-source air drumming system. It uses a smartphone (or PC) to track your arms and motion sensing smart sticks to deliver near zero-latency drum hits.

So, what exactly are you playing? Instead of hitting physical pads, Space Drums uses an AI to dynamically anchor a full, invisible 3D drum kit around your body. The system maps out six distinct, fully playable zones:

1. Hi-Hat: Anchored just to your left.
2. Snare: Positioned right in the center.
3. Floor Tom: Anchored to your right side.
4. Crash & Ride Cymbals: Hovering up top to the left and right.
5. Kick Drum: Tied directly to a sensor in your shoe.

Because the AI tracks your shoulders and joints in real-time, the virtual kit actually adjusts to your position. You get a massive, responsive percussion setup without sacrificing bedroom floor space.

Here is how I over-engineered the ultimate virtual drum kit, and how you can build your own!

Electronics:

1. ESP32-S3 Dev Modules (x3 - Two for sticks, one for the kick pedal): Purchase
2. Adafruit LSM6DS3TR-C IMU sensors or any other IMU (x3): Purchase
3. Lithium-ion batteries (~2000 mah): Purchase
4. USB-C charging modules for standard 5V power supplies: Purchase
5. Power switches: Purchase

Hardware & Materials:

1. 1-inch diameter PVC pipes (for the drumstick bodies)
2. Wood finish vinyl wrap: Purchase

Tools:

1. Soldering iron and solder
2. Wire cutters/strippers

My first idea for this project was simple: put a motion sensor inside a plastic pipe, wire it to a microcontroller, and let it track my drum hits.

To do this, I used something called an IMU (Inertial Measurement Unit). If you've ever wondered how your smartphone knows when you rotate it sideways to watch a video, that's an IMU at work.

The specific sensor I chose (the Adafruit LSM6DS3TR-C) has 6 "Degrees of Freedom," which means it packs two different sensors into one tiny microchip:

1. The Accelerometer (The Hit Detector) An accelerometer measures acceleration and G-forces. When you are air drumming, your stick is moving fast and then suddenly stops when you "hit" the invisible drum. The accelerometer feels that sudden, violent stop. By writing a little bit of code, I could tell the microcontroller: "If you feel a massive spike of negative G-force, that means I just hit a drum! Play a sound!"

2. The Gyroscope (The Direction Finder) Okay, so the accelerometer knows when I hit a drum. But how does the computer know which drum I hit? Is it the snare? The hi-hat? The crash cymbal? That's where the gyroscope comes in. A gyroscope measures rotation and angles. The plan was to start with my sticks pointing straight ahead at the "snare drum." If the gyroscope felt me rotate my wrist 45 degrees to the left, the code would know I was aiming at the hi-hat. If I angled the stick upwards, it would know I was aiming at the crash cymbal.

Overall Architecture:

1. The Accelerometer detects the exact millisecond of the impact.
2. The Gyroscope checks what angle the stick is currently pointing at.
3. The ESP32 microcontroller combines those two pieces of data and instantly plays the correct drum sound over WiFi.

It was foolproof. I wired up my first prototype, wrote the hit-detection logic, and started swinging. The hit detection felt amazing! I was tapping out blast beats in thin air.

But there was a flaw in my plan, which I hadn't yet discovered. More about that soon.

If you're now planning to shove the microcontroller, motion sensor, lithium battery, a charging circuit, and a power switch into a 1-inch plastic tube using just loose wires... well, don't. If you do, expect short circuits and broken connections every time you swing the stick.

To make the drumsticks durable enough to survive intense drumming, I decided to make a custom Printed Circuit Board (PCB).

I booted up EasyEDA (a free circuit design software) and spent hours mapping out how the ESP32-S3 brain would connect to the Adafruit IMU, power switch, and battery. I carefully routed all the copper traces to be as compact as possible, generated the manufacturing files (Gerbers), and sent them off for manufacturing.

A couple of weeks later, they arrived. They looked incredibly professional. I immediately plugged my USB-C cable into my computer to upload the code, and...Nothing. No response from the PC or the PCB.

Looks like when designing the circuit, I made some error with the USB-C port. Without a way to put code on the board, my beautiful PCBs were just expensive paperweights.

Thankfully, during the design phase, my inner mad-scientist self kicked in. Just in case something went wrong, I had added, exposed pins on the edge of the board for the TX (Transmit) and RX (Receive) pins.

Because I exposed those backup pins, the project was saved! Here is how I had to flash the boards (and how you might have to if you ever lock yourself out of a USB port):

1. I grabbed a cheap FTDI USB-to-Serial adapter (a little tool that translates USB data into raw TX/RX signals).
2. I wired the TX pin of the adapter to the RX pad on my board, and the RX pin of the adapter to the TX pad on my board.
3. I manually held down the tiny BOOT button on the ESP32 (which also I added while designing as backup) while applying power to force it into upload mode.
4. I hit "Upload" in the Arduino IDE, and the code successfully flashed!

If you want to build this yourself without dealing with a rat's nest of wires, you can download my exact Gerber files and PCB designs from my github repo: github.com/Arpanmondalz/Space-Drums

So, my prototype sticks were built, the hit detection felt snappy, and I was ready to rock. I sat down, calibrated my forward direction, and started playing. For the first two minutes, it was magical.

But around minute three, something weird happened. To hit the snare drum, I had to angle my hands slightly to the right. A minute later, I had to reach even further right. By minute five, I was practically playing backwards.

I had just come across Gyroscope Drift.

What is Gyro Drift? A gyroscope is great at measuring relative movement (e.g., "I just turned 10 degrees left"). But it has no idea where "True North" or "Forward" actually is. Over time, microscopic math errors build up in the sensor. Slowly, the gyroscope "forgets" where forward is, and your entire virtual drum kit slowly rotates around you while you play.

Failed Attempt #1: The Digital Compass (Magnetometer) The standard engineering fix for drift is adding a Magnetometer (a digital compass) to lock onto the Earth's magnetic field. But before I bought new sensors, I read the reviews of expensive, commercial air drums that use magnetometers. They were full of complaints about drift and frequent calibration needs! If you play drums near a metal desk, a computer tower, or big audio speakers (which have massive magnets in them), the compass gets confused, and the drums still drift.

Failed Attempt #2: The Infrared Anchor I realized I couldn't rely on the Earth's magnetic field; I needed a physical anchor point in my room. I spent weeks building an Infrared (IR) tracking system. I put an IR transmitter on the floor next to my kick drum pedal, and wired IR receivers onto my drumsticks. The idea was that the sticks would constantly look at the floor beacon to re-center themselves. The result was a Total disaster. First, the ESP32’s WiFi radio caused massive electrical interference with the sensitive IR receivers. Second, the ambient light from my living room lamp completely blinded the sensors.

I was staring at my phone, frustrated, when it hit me. I didn't need to build a complex beacon system. I already had the most powerful, high-resolution tracking beacon sitting right in my pocket: a smartphone camera.

Instead of forcing the dumb sticks to guess where they were in the room, what if I used a camera to just look at me?

I threw out the orientation code on the sticks and pivoted to Computer Vision. By running the camera feed through Google's MediaPipe AI (a machine learning model that tracks human posture), the system could instantly map my shoulders, elbows, and wrists in 3D space.

I didn't need the sticks to know where they were anymore. The AI would watch my arms, anchor the invisible drum kit directly to my body, and completely eliminate drift forever.

Now, you might wonder: why not just use the camera for everything and play with regular wooden sticks? The issue with machine learning is latency.

The camera and neural network pipeline run about 150ms behind physical reality. If you hit an air drum and hear the sound half a second later, it's completely unplayable.

To fix this, I built a custom spacial engine:

1. The Reflexes: When a physical drumstick hits the "air," the hardware IMU detects it instantly and fires a signal over WiFi via UDP port 5556.
2. The Time Machine: When the UDP hit arrives, the software grabs the last 50ms of visual tracking data, calculates the stick's terminal velocity, and pushes the trajectory forward in time. It predicts exactly where the stick is going to land, bypassing the visual processing delay.

To run the software: You can use the Android app, or run the Python server on Windows or Linux. (Linux is highly recommended for sub-10ms audio response using ALSA!). You can find the full installation commands for the Python environments in my GitHub Repository.

You need to flash the ESP32-S3 boards using the Arduino IDE. The sticks communicate via UDP to ensure instant data transfer.

1. Add the ESP32 board package to your Arduino IDE preferences.
2. Select "ESP32S3 Dev Module" and ensure USB CDC On Boot is disabled.
3. Open the space_drums_AI.ino sketch.
4. Change the #define STICK_ID to "Left" and upload to your first stick.
5. Change it to "Right" for the second stick.
6. Open spaceDrums_kick.ino and upload it to your shoe module.
7. Tip: If the upload fails, you may need to manually enter Bootloader Mode: Hold the BOOT button, press RESET, release RESET, then release BOOT.

When you boot them up the first time, you will need to connect to the "AirDrums-Stick" WiFi network to add your local network credentials.

I wanted these to feel as close to real drumsticks as possible, which meant fitting all the electronics: the ESP32-S3, the IMU, a 2200mAh lithium battery, a charging module, and a power switch into a standard 1-inch PVC pipe.

This phase took a surprising amount of trial and error because if the sensor rattles inside the stick, it will trigger false hits. Everything had to be rock solid.

1. Preparing the Stick Bodies I cut two lengths of 1-inch diameter PVC pipe to 12 inches long. Bare black PVC looks terrible, so to give them that classic percussion look, I carefully wrapped the pipes in a wood-grain vinyl adhesive.

2. Wiring the Power System Before sliding anything into the tube, I had to wire up the power.

1. I connected the lithium battery to the TP4056 USB-C charging board.
2. I wired a mini slide-switch to the positive output of the charger.
3. I connected the other side of the switch to the VIN and GND pins of my custom ESP32 PCB.

3. The Kick Pedal (The "Shoe" Module) You can't play a drum kit without a bass drum. I assembled a third, identical circuit board, but instead of putting it in a pipe, I put it inside a tiny 3D box.

Because a kick drum doesn't require complex 3D tracking, I didn't need the camera to watch it. I simply added a fabric band to this to slide into any shoe. The firmware for this module (spaceDrums_kick.ino) looks for a sudden spike in downward acceleration (anything over 1.8 Gs) and instantly fires a "KICK" signal over the Wi-Fi.

After three months of pulling my hair out over gyroscope drift, network interference, and latency math, I finally had a completed set of Space Drums sitting on my desk. Here is how you actually use them.

1. Charging

Because we used the TP4056 charging module in the base of the sticks, charging is as simple as plugging a standard USB-C cable into the bottom cap. A red LED will glow inside the tube while it charges, and it will turn blue (or green) when the battery is fully topped up.

2. The First Boot (Network Setup)

The very first time you flick the power switch on the sticks and your shoe module, they won't know how to talk to your computer.

1. Turn on the sticks.
2. Grab your phone or laptop and look for a new WiFi network called "AirDrums-Stick" (and "AirDrums-Kick" for the shoe).
3. Connect to it, and a login portal will pop up.
4. Enter your home WiFi network name and password. The sticks will save this forever! They will automatically connect to your home network from now on! If they don't find the network, they will open the portal again for you to add the new WiFI details.

3. Starting the AI Server

You can run the server via the Android App, or run the Python script on your PC (Linux is highly recommended for the lowest audio latency!).

1. Start the server (e.g., run python main.py on your computer).
2. The terminal will spit out a local IP address (like http://192.168.1.13:5000).
3. Open that link on your smartphone's web browser, grant it camera permissions, and prop your phone up so it can see your upper body.

As soon as the server is running, the sticks will automatically find it over the network.

4. Calibration

After power on, hold both sticks parallel in front of you (pointing at where a snare drum would be), and wait for about 5 seconds. The firmware will detect and offset any spikes and drifts in the IMU data.

The computer vision model will lock onto your shoulders, elbows, and wrists, and generate the invisible drum kit around you.

You are now ready to rock. Swing a stick, and the accelerometer will detect the massive negative G-force of the "hit," fire a signal over UDP, and combine with the AI's velocity prediction to play a sound instantly.

Because the AI anchors the kit to your body, the zones are incredibly intuitive:

1. Hi-Hat: Floating just to your left.
2. Snare: Positioned right in the center.
3. Floor Tom: Anchored to your right side.
4. Crash & Ride Cymbals: Hovering up top.
5. Kick Drum: Stomp that right foot!

So, what does it actually feel like? I'll be brutally honest. If you are a professional drummer, this will not replace a $2,000 physical acoustic kit. You don't get the physical rebound of a wooden stick bouncing off a tight mylar drumhead.

But for someone who wants to learn coordination, casually jam with friends, or play a full drum set in a tiny apartment at 2 AM without getting evicted, it is absolutely incredible. It feels like magic!

We live in a world of next-day delivery. If I really wanted to, I could have just thrown a few hundred dollars at a commercial electronic drum set, and it would have arrived on my doorstep in a day or two. It would have been faster, easier, and saved me three months of debugging gyroscope drift, network interference, and latency math.

But here is the thing about buying consumer electronics: it is just a transaction. When you swipe your card, you get a polished product, but you don't build any real connection with it.

When you build something yourself, when you solder the tiny components, debug hundreds of lines of code, and finally beat the physics problems that kept you awake at 2 AM, you understand that object on a fundamental level. Every time I hit an air drum and hear that instant crash cymbal, it isn't just a sound, I feel the signals traveling through the PCB traces, microcontroller doing to math and the physics engine predicting where exactly I hit, all in less than a hundred milliseconds.

If you decide to build your own Space Drums and experience that feeling for yourself, let me know in the comments! All the code, 3D files, circuits, and audio samples are completely open-source and waiting for you on my GitHub Repository.

Thanks for STICKing along!