Leaf Touch Symphony: a Plant-Interactive MIDI Music Device

Make music by touching houseplants — no musical experience required.

Leaf Touch Symphony (叶触交响曲) is a real-time music generation device that uses living plants as its only interface. Touch a leaf, make music. Seven leaves, seven notes. No keyboard, no screen, no skill needed — just your fingers and a plant.

Built on an ESP32 microcontroller with I2S digital audio output, the device runs entirely standalone — no computer, no Wi-Fi required. It features two play modes, pressure-sensitive volume mapping, an OLED display, and a 3D-printed grand piano enclosure.

⭐ [INSERT DEMO VIDEO HERE — upload your video file to this step in Instructables]

How It Works

The ESP32 has built-in capacitive touch sensing pins. When you touch a plant leaf connected to one of these pins, your body, the plant, and the ground form a small capacitive circuit — enough for the ESP32 to detect the touch and measure its intensity.

The firmware maps that touch intensity continuously to audio volume (velocity), so pressing harder makes the sound louder — just like a real instrument. Audio is synthesized in real-time as pure sine waves, mixed digitally, and sent via I2S to a MAX98357A amplifier board that drives a speaker.

Two Play Modes

Mode Description

Mode 1 – Free Play

Each of the 7 leaves plays one note of a C-major scale (C4–B4). Touch multiple leaves simultaneously for chords and polyphony. Touch harder for louder sound.

Mode 2 – Song Mode

Each leaf triggers a complete preset melody (Twinkle Twinkle, Ode to Joy, scales, etc.). Edge detection prevents the song from restarting while you hold the leaf.

Switch between modes from the main menu shown on the OLED display. After 150 seconds of no touch, the device automatically returns to the menu — perfect for unattended exhibition setups.

Project Overview

1. Microcontroller: ESP32 (38-pin dev board)
2. Audio output: I2S → MAX98357A amplifier → 4Ω 3W speaker
3. Display: SSD1306/SSD1315 OLED 128×64 (I2C)
4. Touch input: 7 capacitive touch GPIO pins, one wire per plant leaf
5. Enclosure: 3D-printed grand piano shell (PLA+ or PLA+Wood)
6. Power: 5V USB — fully standalone, no internet required
7. Build time: 8–12 hours (excluding 3D print time)
8. Estimated cost: ~$25–40 USD in components
9. Difficulty: Intermediate

Here is everything you need to build Leaf Touch Symphony. Most components are available from AliExpress, Amazon, or your local electronics supplier.

Electronics

ESP32 Development Board — ESP32-S (38-pin), qty: 1

Must have capacitive touch pins (T0–T9).

I2S Amplifier Module — MAX98357A, qty: 1

3W output, I2S digital input — no external DAC chip needed.

OLED Display — SSD1315 or SSD1306, 128×64, I2C, qty: 1

0.96 inch, 4-pin I2C version.

Speaker — 4Ω 3W, qty: 1

Sized to fit inside the 3D-printed piano lid.

Jumper Wires (Dupont) — male-to-male and male-to-female, qty: ~30

For internal connections between all modules.

USB Cable — Micro-USB or USB-C (match your ESP32 board), qty: 1

For programming and power.

USB Power Supply — 5V, ≥1A, qty: 1

Powers the entire device during operation.

Alligator Clip Wires — any color, ~20–30 cm length, qty: 7

For clipping onto plant leaf stems. Optional but strongly recommended over bare wires.

Plant

Potted houseplant

Choose one with at least 7 large, reachable leaves. Best choices: Pothos, Peace Lily, Rubber Plant, Philodendron. Keep soil slightly moist for best conductivity.

Backup plants (optional)

2–3 identical plants recommended for exhibition use, in case one dries out during a long event.

Tip: Hydroponic (water-grown) plants give the most consistent touch readings and are ideal for exhibitions.

3D Printing & Enclosure

FDM 3D Printer

Capable of printing PLA or PLA+ filament.

PLA+ or PLA+Wood filament — ~300–400g estimated total

Wood-fiber PLA gives a warmer, piano-like aesthetic.

Prop rod / support post — ~8 cm long

Holds the piano lid open at 45°. Can be 3D printed as a simple cylinder or use a wooden dowel.

M3 screws and nuts

For lid hinge and body assembly.

Superglue or hot glue gun

For fixing internal components and sealing body segments.

Tools

1. Soldering iron + solder (for MAX98357A header pins if not pre-soldered)
2. Wire stripper
3. Small flathead screwdriver
4. Multimeter (optional, useful for debugging wiring issues)
5. Computer with Arduino IDE installed (for firmware upload)

Before wiring anything, it helps to understand how the four layers of the system work together to turn a leaf touch into music.

Layer 1 — Sensing Layer

Reads capacitive touch from each of the 7 leaves and maps contact intensity to audio volume.

1. Leaf 1–4: GPIO4 / GPIO27 / GPIO33 / GPIO32
2. Leaf 5–7: GPIO14 / GPIO13 / GPIO15
3. Uses ESP32's built-in touchRead() API
4. Each leaf has its own calibrated trigger threshold
5. map() converts raw reading → velocity 0–255

Layer 2 — Processing Layer (ESP32 Main Control)

Runs the state machine and handles all logic.

1. State machine: mode_select / do_mode1 / do_mode2
2. Edge detection via prevTouch[] array (prevents songs from restarting on hold)
3. 150-second idle auto-return timer
4. Serial debug output every 500ms for calibration

Layer 3 — Synthesis Layer

Mode 1: 7 sine wave oscillators running in parallel. Each oscillator maintains its own phase independently, so multiple leaves can sound simultaneously (polyphony). Output is summed and divided by 7, then scaled to 16-bit integer.

Mode 2: A timing engine reads MIDI note arrays and uses millis() to advance through notes at fixed durations. Oscillator phase is reset to 0 at the start of each song to prevent click artifacts.

Layer 4 — Output Layer

1. I2S digital audio: 22050 Hz / 16-bit / mono → MAX98357A → 4Ω 3W speaker
2. OLED display (SSD1315, I2C, 128×64): shows current mode name and song info

Main Loop Design — Audio First

The loop() function always processes audio before anything else, guaranteeing smooth uninterrupted sound:

1. Call fillAudioBlock() — synthesize 64 audio samples into the buffer (~2.9 ms of audio)
2. Call i2s_write() — send buffer to I2S DMA, wait until confirmed sent
3. Read all 7 touch pin values
4. Run current state handler (do_mode_select / do_mode1 / do_mode2)

This gives a touch sensing latency of ~3 ms — imperceptible to the human hand and ear.

Touch Sensing — How the Capacitive Circuit Works

When you touch a leaf, your body + the plant + the earth form a tiny capacitive loop. The ESP32's touchRead() returns a number that drops when touched (more capacitance = lower reading). The firmware then processes it through these steps:

1. Raw read: touchRead(BeTouch1) — returns 5–60 at idle, drops sharply on touch
2. Trigger check: reads[i] <= ybetouched[i] — per-leaf threshold array: {10, 10, 10, 10, 11, 13, 12}
3. Velocity map: map(reads, 0, threshold, 255, 0) — lower reading = stronger touch = louder sound
4. Clamp: constrain(reads, 0, threshold) — prevents out-of-range values from crashing map()
5. Edge detection (Mode 2 only): current[i] && !prevTouch[i] — triggers song on press, not hold
6. Idle timeout: millis() - lastTouchTime > 150000 — returns to menu after 150s

I2S Audio Configuration

1. Sample Rate: 22050 Hz — covers full human hearing range with moderate memory use
2. Bit Depth: 16-bit (int16) — 96 dB dynamic range
3. Channel: Mono (left only) — saves CPU; single speaker anyway
4. DMA buffers: 8 × 64 samples = ~23 ms of audio headroom to prevent dropouts
5. Block size: 64 samples per loop iteration = ~2.9 ms, main loop runs at ~345 Hz
6. tx_desc_auto_clear: true — auto-clears I2S queue when full, preventing main loop stalls

All connections reference the ESP32-S 38-pin development board. Wire up the three modules below in any order — just double-check each connection against the lists before powering on.

OLED Display (SSD1315 / SSD1306)

1. GND → GND
2. VCC → 3V3
3. SCL → GPIO18 (I2C clock)
4. SDA → GPIO19 (I2C data)

Default I2C address is 0x3C. If nothing shows on the display, try 0x3D in the firmware's display.begin() call.

7 Leaf Touch Inputs

1. Leaf 1 (T1) → GPIO4 — plays C4 (Do) / triggers Twinkle Twinkle
2. Leaf 2 (T2) → GPIO27 — plays D4 (Re) / triggers Ode to Joy
3. Leaf 3 (T3) → GPIO33 — plays E4 (Mi) / triggers C Major Scale
4. Leaf 4 (T4) → GPIO32 — plays F4 (Fa) / triggers Pentatonic Scale
5. Leaf 5 (T5) → GPIO14 — plays G4 (Sol) / triggers Rhythmic Pattern
6. Leaf 6 (T6) → GPIO13 — plays A4 (La) / triggers G Major Fragment
7. Leaf 7 (T7) → GPIO15 — plays B4 (Ti) / triggers Children's Tune

Run a jumper wire from each GPIO pin to the plant using one of these contact methods:

1. Alligator clip on the leaf stem — most reliable, recommended
2. Metal probe or paperclip inserted in the soil near the leaf's root — also works well
3. Bare wire touching the underside of the leaf — works but less consistent

⚠️ Important: These are sense-only pins. Do NOT connect them to any voltage source. They connect only to the plant wire.

MAX98357A I2S Amplifier

1. BCLK → GPIO26 (I2S bit clock)
2. LRC → GPIO25 (I2S word select / left-right clock)
3. DIN → GPIO22 (I2S data input)
4. GND → GND (shared ground with ESP32)
5. VIN → 5V — NOT 3.3V! Use the ESP32 board's VIN/5V pin (available when powered by USB)
6. SD (Shutdown) → 3V3 (pull high to enable the chip)
7. GAIN → leave floating or connect to 3V3 (sets amplifier gain — check your module's datasheet)
8. SPK+ / SPK– → Speaker positive / negative terminals

⚠️ Critical: The MAX98357A must be powered from 5V. Powering from 3.3V will produce very low volume or no output.

Recommended Wire Color Convention

1. Black — GND
2. Red — 3.3V power
3. Orange — 5V power
4. Yellow — I2S BCLK (GPIO26)
5. Green — I2S LRC (GPIO25)
6. Blue — I2S DIN (GPIO22)
7. Purple — I2C SCL (GPIO18)
8. White — I2C SDA (GPIO19)
9. Other colors ×7 — one per leaf touch input

Label each leaf wire L1–L7 with tape or a marker. You'll thank yourself during assembly.

The firmware is a single Arduino sketch (.ino file) that handles everything: touch sensing, audio synthesis, song playback, OLED display, and the state machine.

Key Firmware Parameters

1. SAMPLE_RATE = 22050 Hz — audio sample rate, covers full human hearing range
2. BLOCK_SIZE = 64 samples — audio buffer per main loop iteration (~2.9 ms each)
3. ybetouched[] = {10, 10, 10, 10, 11, 13, 12} — touch trigger threshold per leaf (calibrate for your plant)
4. DMA buffers = 8 × 64 samples — ~23 ms of audio headroom to prevent dropouts
5. Idle timeout = 150,000 ms — auto-returns to main menu after 150 seconds of no touch
6. I2S pins: BCLK → GPIO26, LRC → GPIO25, DIN → GPIO22

Mode 1 — Note Mapping (Free Play)

The 7 leaves play a C-major scale using the standard MIDI pitch formula:

freq = 440 × 2^((note − 69) / 12)

1. Leaf 1 — MIDI 60 — C4 (Middle C) — 261.6 Hz — oscillator: osc1_phase
2. Leaf 2 — MIDI 62 — D4 — 293.7 Hz — oscillator: osc2_phase
3. Leaf 3 — MIDI 64 — E4 — 329.6 Hz — oscillator: osc3_phase
4. Leaf 4 — MIDI 65 — F4 — 349.2 Hz — oscillator: osc4_phase
5. Leaf 5 — MIDI 67 — G4 — 392.0 Hz — oscillator: osc5_phase
6. Leaf 6 — MIDI 69 — A4 (Standard A) — 440.0 Hz — oscillator: osc6_phase
7. Leaf 7 — MIDI 71 — B4 — 493.9 Hz — oscillator: osc7_phase

Each oscillator runs this logic per sample inside fillAudioBlock():

1. Compute sample: sin(phase) × velocity / 255
2. Advance phase: phase += 2π × freq / 22050, wrap at 2π to prevent float overflow
3. Sum all 7 oscillators and divide by 7.0 to avoid clipping
4. Scale to int16: multiply by 32767, clamp to [−32768, 32767]
5. Write to I2S buffer via i2s_write()

Mode 2 — Preset Song Sequences

1. Leaf 1 — Twinkle Twinkle (opening) — notes: 60 60 67 67 69 69 67
2. Leaf 2 — Ode to Joy (adapted) — notes: 64 64 65 65 64 64 62
3. Leaf 3 — C Major Scale ascending — notes: 60 62 64 65 67 69 71
4. Leaf 4 — Pentatonic Scale descending — notes: 67 65 64 62 60
5. Leaf 5 — Rhythmic Pattern — notes: 60 62 60 62 60 62 64
6. Leaf 6 — G Major Fragment ascending — notes: 67 69 71 72 74
7. Leaf 7 — Children's Tune — notes: 62 64 65 67 65 64 62

Song timing engine:

1. startSong() records a start timestamp and resets oscillator phase to 0 (prevents click sounds)
2. updateSong() uses millis() to advance to the next note at the right time
3. Each note duration: 300–1000 ms. Final note is longer to create a natural ending feel
4. When song finishes: isPlayingSong = false, songFreq = 0 — output silences naturally

Uploading the Sketch

1. Open the .ino file in Arduino IDE
2. Verify board settings match Step 4
3. Click the Upload button (→ arrow icon)
4. If upload fails to start, hold the BOOT button on the ESP32, then click Upload, then release BOOT
5. Wait for "Done uploading" in the console
6. Open Tools → Serial Monitor at 115200 baud — touch readings should print every 500 ms

Different plants have different conductivity depending on species, soil moisture, and room humidity. You'll need to adjust the ybetouched[] array in the firmware for your specific plant. This takes about 5 minutes.

Why Calibration Is Necessary

The ESP32's touchRead() values vary based on:

1. Plant species — some plants conduct electricity better than others
2. Soil moisture — wetter soil means lower idle readings
3. Wire length — longer wires pick up more environmental noise
4. Room humidity — higher humidity shifts idle values
5. Contact point — stem clips are more consistent than soil probes
6. 
   

Calibration Procedure

1. Power on the device and connect your computer via USB
2. Open Arduino IDE → Tools → Serial Monitor → baud rate: 115200
3. The firmware prints raw touch readings every 500 ms:
4. T1:35 T2:28 T3:41 T4:33 T5:22 T6:18 T7:29
5. Without touching any leaves, note the idle values for each pin
6. Touch each leaf firmly and note the values when touched (they will drop significantly)
7. Set each ybetouched[i] to roughly halfway between the idle and touched values
8. Re-upload the firmware with updated thresholds
9. Test all 7 leaves — each should trigger cleanly without false positives

Example Calibration Session

Based on a pothos plant with moist soil and 20 cm wires:

1. Leaf 1: idle 35, touched 4 → set threshold to 15
2. Leaf 2: idle 28, touched 5 → set threshold to 13
3. Leaf 3: idle 41, touched 7 → set threshold to 20
4. Leaf 4: idle 33, touched 6 → set threshold to 16
5. Leaf 5: idle 22, touched 8 → set threshold to 13
6. Leaf 6: idle 18, touched 9 → set threshold to 12
7. Leaf 7: idle 29, touched 5 → set threshold to 14

Default firmware values {10, 10, 10, 10, 11, 13, 12} are tuned for a pothos with moist soil and short wires. Adjust as needed.

Troubleshooting Touch Issues

Leaf triggers without being touched:

Threshold is too high, or a power cable is causing noise. Lower ybetouched[i] by 2–3 counts. Keep leaf wires away from USB and speaker cables.

Leaf never triggers even with firm touch:

Threshold is too low, or poor contact. Raise ybetouched[i]. Water the plant. Move wire contact closer to the leaf base or stem.

One leaf works, adjacent leaf doesn't:

Check the physical connection at both ends — the GPIO pin on the ESP32 and the plant contact point.

All readings are very high (50+) even when touched:

Plant is too dry. Water thoroughly and wait 30 minutes, then retest.

Tips for Consistent Sensing

1. Keep soil slightly moist — not waterlogged, but never bone dry
2. Hydroponic plants give the most consistent readings — ideal for exhibitions
3. Use alligator clips on stems rather than probes in soil
4. Keep leaf wires away from power cables (USB wire, speaker wire) to reduce noise
5. Re-calibrate if you move to a different room or humidity changes significantly
6. For exhibitions: calibrate the day before the event and have 2–3 backup plants ready

The device is housed in a miniature 3D-printed grand piano shell. The design creates an intuitive metaphor: leaves = piano keys. Opening the piano lid signals "ready to play" — and that's where the plant sits.

Design Concept

We chose a grand piano shape (rather than upright) for two practical reasons:

1. The openable lid creates a natural storage space for the electronics inside
2. Opening the lid is a physical gesture that signals "the instrument is ready" — it gives the interaction a sense of ritual and invitation

Lid Dual-State Design

Closed state: Compact piano shape, electronics protected, clean exterior. Use case: Storage, transport, or decorative display when not in use.

Open state (45°): Lid propped open by a support post; plant and electronics visible. Use case: Active play mode; also exposes USB port for firmware updates.

Component Layout Inside the Piano

1. Piano body (bottom): Houses the ESP32 dev board, MAX98357A amplifier, and all Dupont wiring
2. Piano lid (top): Speaker mounts on the inside of the lid, cone facing outward for best projection
3. OLED display: Mounted on the front face of the piano body, visible when the lid is open
4. Leaf wires: Exit through a small notch or channel at the rear of the body
5. USB port: Accessible from the rear or side of the body for power and programming

Print Settings

Filament: PLA+ (stronger) or PLA+Wood (warmer aesthetic)

Layer height: 0.2 mm

Infill: 20% — gyroid or grid pattern

Supports: Yes — needed for the lid hinge area and piano legs

Print temperature: 200–210°C (PLA+)

Bed temperature: 60°C

Print orientation: Body flat on bed; lid printed separately flat

Printed Parts List

1. Piano body — main shell (can be printed in 2–3 segments and glued if it doesn't fit your print bed)
2. Piano lid — hinged top cover
3. Piano legs × 3 — attach to base of body with M3 screws
4. Prop rod — ~8 cm support post to hold the lid at ~45° when open (can also use a wooden dowel)
5. OLED bracket (optional) — snap-fit holder for the 0.96" OLED module

Assembly Notes

1. If printing in segments, join body sections with superglue or M3 bolt inserts and nuts
2. Leave a cable routing channel along the inside base before gluing segments together
3. Hot-glue the speaker to the inside of the lid, cone facing outward
4. Run speaker cables through the hinge area with enough slack to open and close the lid freely (~10–12 cm extra)
5. Sand any rough layer lines on visible exterior surfaces with 220-grit sandpaper for a smoother finish
6. Optional: paint with acrylic paint for a black lacquer piano look, or leave the wood-fiber PLA natural

With all parts printed and the firmware uploaded, it's time to put everything together. Work through these steps in order — it's much easier to test as you go than to troubleshoot after everything is glued shut.

Assembly Sequence

1. Prepare the Base Unit

Place the ESP32 development board in the bottom of the piano body. Secure it with double-sided tape or 3D-printed standoffs. Route all Dupont cables neatly toward the back of the enclosure, away from the OLED viewing area.

2. Mount the Amplifier

Fix the MAX98357A board next to the ESP32, on either side. Keep the speaker output wires short — under 15 cm if possible. Secure with hot glue or double-sided tape.

3. Mount the OLED Display

Position the OLED on the front face of the piano body so it is clearly visible when the lid is open. If your body has a snap-fit bracket, click it in. Otherwise, use a small amount of hot glue around the edges of the module (not on the screen).

4. Mount the Speaker in the Lid

Apply hot glue around the speaker frame and press it onto the inside face of the lid, cone facing outward (toward the listener). Run the two speaker wires through the hinge area of the lid into the body, leaving enough slack (~10–12 cm) so the lid can open and close freely without straining the wires.

5. Route the Leaf Wires

Run all 7 leaf touch wires from the ESP32 GPIO pins out through a small notch or hole in the rear of the piano body. Each wire should end in an alligator clip or bare metal probe. Label each wire L1 through L7 with tape or a marker.

6. Internal Wiring Check

Before closing any panels, verify all internal connections one more time against the wiring table in Step 3. Pay special attention to:

1. MAX98357A VIN connected to 5V (not 3.3V)
2. All GND lines shared between ESP32, OLED, and MAX98357A
3. I2S pins (GPIO26, GPIO25, GPIO22) connected correctly to MAX98357A
4. I2C pins (GPIO18, GPIO19) connected to OLED SCL and SDA

7. Power On and Test (Before Closing)

With the lid open and all components accessible, plug in the USB power supply and verify:

1. ✅ OLED lights up and shows the main menu
2. ✅ Serial Monitor shows 7 touch readings updating every 500 ms
3. ✅ Touching each leaf in Mode 1 produces the correct note
4. ✅ Touching each leaf in Mode 2 triggers the correct melody
5. ✅ Volume responds to touch pressure (harder = louder)
6. ✅ After 150 seconds of no touch, device returns to main menu

8. Attach the Piano Legs

Screw or snap-fit the three piano legs onto the bottom of the body. Make sure the piano sits level.

9. Place the Plant

Set the potted plant next to or partially inside the open piano body. Clip or insert the 7 leaf wires onto the corresponding leaves. The plant can sit:

1. Beside the piano — easiest setup, looks natural
2. On top of the piano body — compact, dramatic visual
3. Partially inside the open lid — most theatrical, reinforces the "plant growing from the piano" aesthetic

10. Final Check

Touch each leaf one more time with the plant in its final position to confirm all 7 connections are working. Adjust any wire clips that have shifted during assembly. You're done!

Exhibition Setup Tips

1. Position the plant so all 7 wired leaves are easily reachable by visitors without bending
2. Place a small card in front of the device explaining "Touch a leaf to make music"
3. The 150-second idle auto-return means the device self-resets between visitors — no supervision needed
4. Power from a USB power bank for truly wireless, cord-free exhibition setups
5. Keep a backup potted plant nearby; swap it in if the primary plant's leaves dry out during a long event

Leaf Touch Symphony is designed so that anyone can pick it up and make music immediately — no instructions needed. But here's how it works for those who want to explore all its features.

Turning On

Plug in the USB power supply (or USB power bank). The OLED display will light up within 2 seconds and show the main menu with two mode options.

Mode 1 — Free Play

1. From the main menu, select Mode 1
2. The OLED shows "Mode 1 – Free Play"
3. Touch any leaf to hear that leaf's note
4. Touch multiple leaves at once for chords — all 7 oscillators run in parallel
5. Press harder for a louder sound; touch lightly for a softer sound
6. The 7 leaves play a C-major scale: Do – Re – Mi – Fa – Sol – La – Ti

Leaf mapping (Mode 1):

1. Leaf 1 — C4 (Middle C) — Do
2. Leaf 2 — D4 — Re
3. Leaf 3 — E4 — Mi
4. Leaf 4 — F4 — Fa
5. Leaf 5 — G4 — Sol
6. Leaf 6 — A4 — La
7. Leaf 7 — B4 — Ti

Tip: Try leaves 1 + 3 + 5 (C + E + G) together for a C major chord. Or leaves 2 + 4 + 6 (D + F + A) for a D minor chord.

Mode 2 — Song Mode

1. From the main menu, select Mode 2
2. The OLED shows the song name assigned to each leaf
3. Touch any leaf once — the full preset melody plays automatically
4. Hold the leaf while the song plays; the song won't restart until you release and touch again
5. Touch a different leaf to switch to a different song

Song mapping (Mode 2):

1. Leaf 1 — Twinkle Twinkle Little Star (opening phrase)
2. Leaf 2 — Ode to Joy (adapted)
3. Leaf 3 — C Major Scale (ascending)
4. Leaf 4 — Pentatonic Scale (descending)
5. Leaf 5 — Rhythmic Pattern
6. Leaf 6 — G Major Fragment (ascending)
7. Leaf 7 — Children's Tune

Auto-Return (Exhibition Mode)

If no leaf is touched for 150 seconds (2.5 minutes), the device automatically returns to the main menu. This makes Leaf Touch Symphony safe for unattended exhibitions — the next visitor always starts from a clean state.

Switching Modes

To switch between Mode 1 and Mode 2, let the device idle for 150 seconds (it returns to the main menu automatically), or use whatever navigation gesture you've mapped in the firmware for your setup.

If something isn't working as expected, use this guide to diagnose and fix the issue. Work through the relevant section systematically — most problems come down to wiring, power, or touch thresholds.

Audio Problems

Problem: No sound at all

Likely cause: MAX98357A not powered correctly, or I2S wiring error.

Solution: Check that MAX98357A VIN is connected to 5V, not 3.3V. Verify BCLK→GPIO26, LRC→GPIO25, DIN→GPIO22. Check speaker wires aren't reversed.

Problem: Very quiet output

Likely cause: MAX98357A powered from 3.3V instead of 5V.

Solution: Move VIN to the 5V rail. Volume is proportional to supply voltage on the MAX98357A.

Problem: Audio distortion / clipping

Likely cause: Mixer not dividing by 7 before int16 conversion.

Solution: Ensure fillAudioBlock() divides the sum of all oscillators by 7.0 before multiplying by 32767.

Problem: Clicking or popping between notes (Mode 2)

Likely cause: Phase discontinuity when switching notes.

Solution: Confirm startSong() resets oscMode2_phase = 0 before starting each new note or song.

Problem: Audio cuts out / stutters

Likely cause: I2S DMA queue overflow.

Solution: Confirm tx_desc_auto_clear = true in the I2S config. Also check no blocking code is delaying the main loop.

Problem: Device freezes after playing for a while

Likely cause: Stack overflow or heap fragmentation.

Solution: Reduce DMA buffer count, or increase ESP32 stack size. Check for memory leaks in song data arrays.

Display Problems

Problem: OLED shows nothing (backlight off)

Likely cause: No power to OLED.

Solution: Check VCC → 3V3 and GND → GND connections.

Problem: OLED powers on but shows garbage or nothing

Likely cause: Wrong I2C address.

Solution: Try changing 0x3C to 0x3D in display.begin(). You can also run an I2C scanner sketch to find the address.

Problem: OLED shows correct text but freezes

Likely cause: SCL/SDA wiring issue.

Solution: Confirm SCL → GPIO18 and SDA → GPIO19. Make sure no other device is pulling these lines.

Touch Sensing Problems

Problem: Leaf triggers constantly without touch

Likely cause: Threshold too high, or interference from nearby power cables.

Solution: Lower ybetouched[i] by 2–3 counts. Move leaf wires away from USB or speaker cables.

Problem: Leaf never triggers even with firm touch

Likely cause: Threshold too low, dry plant, or poor wire contact.

Solution: Raise ybetouched[i]. Water the plant. Re-clip the wire closer to the leaf base or stem.

Problem: Mode 2 songs keep restarting while holding a leaf

Likely cause: Edge detection not working.

Solution: Check that prevTouch[] is being updated at the end of each loop iteration after edge detection.

Problem: Some leaves work, others don't

Likely cause: Disconnected wire or wrong GPIO pin.

Solution: Check the physical connection at both ends (ESP32 pin and plant contact). Verify the pin assignment matches the wiring table.

Upload / Firmware Problems

Problem: ESP32 not detected as COM port

Likely cause: Missing USB-to-Serial driver.

Solution: Install the CH340 or CP2102 driver for your OS. Check which chip your ESP32 board uses (usually printed on the board near the USB connector).

Problem: "Failed to connect to ESP32" error during upload

Likely cause: ESP32 not in bootloader mode.

Solution: Hold down the BOOT button on the ESP32, click Upload in Arduino IDE, then release BOOT once you see "Connecting..." in the console.

Problem: Sketch compiles but behavior is wrong

Likely cause: Old firmware still running.

Solution: Try holding BOOT + pressing EN (reset) together, then release EN first, then BOOT. This forces the ESP32 into download mode.

Leaf Touch Symphony is designed to be hacked. All the interesting parameters are clearly labeled in the firmware, so even a beginner can make meaningful changes. Here are the most useful things to try.

Easy Customizations (No Hardware Changes)

Change the Notes in Mode 1

Edit the MIDI note numbers in the oscillator frequency array. For example, swap the C-major scale for a pentatonic scale — it's almost impossible to play a bad-sounding combination:

Try: {60, 62, 64, 67, 69, 72, 74} — C pentatonic, two octaves

Or: {60, 63, 65, 67, 70, 72, 75} — C minor pentatonic

Add More Songs in Mode 2

Each song is just an array of MIDI note numbers and an array of note durations in milliseconds. To add a new song:

1. Create a new note array: int mySong[] = {64, 62, 60, 62, 64, 64, 64};
2. Create a duration array: int mySongDur[] = {400, 400, 400, 400, 400, 400, 800};
3. Add it to the song selection logic and assign it to a leaf

Speed Up or Slow Down Songs

Change the per-note duration values in the song timing arrays. Halving all durations doubles the tempo. The last note in each song is typically set longer (800–1000 ms) to create a natural ending — keep that pattern.

Change Touch Sensitivity

Adjust ybetouched[] thresholds as described in Step 6. Lower values = require a firmer touch; higher values = respond to lighter touches.

Change the Idle Timeout

Change 150000 in the timeout check to any millisecond value. Set it to 30000 for a 30-second timeout, or 300000 for 5 minutes.

Intermediate Customizations

Richer Timbre — Add Harmonics

The current firmware uses pure sine waves, which sound clean but thin. To make the sound richer, add the second harmonic (2× the frequency at half the amplitude):

sample = sin(phase) * 0.7 + sin(phase * 2) * 0.3;

This gives a slightly hollow, flute-like tone. Adding a third harmonic gives a more organ-like quality.

ADSR Envelope

Currently, notes start and stop instantly. Implementing a simple attack-decay-sustain-release (ADSR) envelope makes notes feel more natural and musical. At minimum, a short attack (5–10 ms fade-in) and release (20–50 ms fade-out) eliminates any remaining click artifacts.

Smarter Mixer

The current mixer always divides by 7, even when only 1 or 2 oscillators are active — losing volume unnecessarily. Change it to divide by the count of active oscillators:

sum / max(activeCount, 1)

Advanced Extensions (Project Roadmap)

v1.2 — Bluetooth MIDI Output

Use ESP32's built-in BLE to send standard MIDI Note On/Off + Velocity messages to a phone running GarageBand, or any BLE MIDI-compatible DAW. Your plant becomes a real MIDI controller.

v2.0 — Scale Presets

Add a physical rotary switch to select different scales (pentatonic, jazz, blues, chromatic) and rhythmic modes. No firmware re-upload needed — the switch changes the mode at runtime.

v2.5 — Multi-Plant Ensemble

Use multiple plants, each connected to its own ESP32 or set of touch pins. One plant controls melody, one controls chords, one controls rhythm — a full plant orchestra.

v3.0 — AI Accompaniment

Connect to a cloud NLP API (when Wi-Fi is added). Voice or text commands switch songs and timbres. The device generates melody fragments based on natural language descriptions like "play something melancholy."

v4.0 — Web Mood Interface

A lightweight HTML page + AI model lets users "talk to the plant." The model reads the user's emotional state and selects music accordingly — a truly empathetic plant instrument.

Share Your Version!

If you build Leaf Touch Symphony or customize it in an interesting way, share your photos and variations in the comments below. We'd especially love to see:

1. Different plant species and how they affect touch sensing
2. Custom enclosure designs beyond the grand piano shape
3. New song sets or scale modes added to the firmware
4. Bluetooth MIDI setups connecting to a DAW

Leaf Touch Symphony was created by a student innovation team, supervised by Associate Professor Chen Shichao (Plant Systematics & Molecular Evolution).

Team

Project Lead — Xie Yaru (解雅茹) 40%

Overall system design, firmware development, hardware testing

Member — Liu Kewei (刘柯微) 30%

Hardware assembly, soldering, component testing

Member — Wang Zi (王紫) 30%

Enclosure design, 3D modeling, final assembly

Faculty Advisor: Associate Professor Chen Shichao — Plant Systematics & Molecular Evolution. Provided botanical expertise and innovation competition mentorship throughout the project lifecycle.

Project Background

This project was motivated by three gaps we saw in existing interactive music devices:

1. High barrier to entry — most require musical training or DAW software knowledge
2. Artificial interfaces — keyboards and touchscreens lack the natural, tactile quality of interacting with a living thing
3. High cost — commercial interactive installations typically cost thousands of dollars, making them inaccessible to students and hobbyists

Leaf Touch Symphony addresses all three: zero musical knowledge needed, a living plant as the interface, and a total component cost under $40.

The underlying capacitive sensing technique — using the human body + plant + ground as a capacitive circuit — was first validated by Disney Research's Touché project (2012). Our contribution is combining this with a fully embedded, standalone real-time audio synthesis engine running on the ESP32, with no external computer required.

References

1. Sato M., Poupyrev I., Harrison C. Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. Proceedings of the 30th ACM SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2012.
2. Espressif Systems. ESP32 Technical Reference Manual v5.2. 2024.
3. Espressif Systems. ESP32 I2S Driver API Reference. https://docs.espressif.com/projects/esp-idf
4. Best F. Arduino MIDI Library Documentation. 2022.
5. Adafruit Industries. Adafruit SSD1306 Library Documentation.
6. MIDI Manufacturers Association. Standard MIDI Files Specification 1.0.
7. Bare Conductive. TouchBoard & Plant Instrument Open Source Cases and Technical Forums.

Let every leaf become a doorway to music.

Tags: ESP32, MIDI, music, capacitive touch, plants, Arduino, I2S, interactive art, HCI, synthesizer, DIY, maker, embedded audio