DIY AI Pin: a Wearable Vision/Voice Assistant

In this project, you will learn how to build your own DIY AI Pin, a wearable vision and voice assistant based on the Xiao ESP32S3 Sense board. This compact device can be worn on your chest and works as a personal assistant that can “see” through its camera and “hear” through its microphone.

The AI Pin can identify objects, answer general knowledge questions, and even interact with smart home devices. For example, you can ask it how you look, or identify objects like camera lenses. It can also integrate with Home Assistant to control devices like lights using simple voice commands.

Main Electronics:

1. XIAO ESP32-S3 Sense — Main controller with Wi-Fi, microphone, and camera support
2. MAX98357A I2S Amplifier — Converts digital audio from the Xiao S3 into speaker output
3. 8Ω 2W Ultra-Thin Cavity Speaker — Compact speaker used for voice responses and audio playback
4. 3.7V Li-Po Battery — Portable power source

PCB & Assembly:

1. Custom PCB (or Breadboard for prototype)
2. Pin headers
3. Jumper wires
4. Mounting screws & standoffs

Enclosure:

1. 3D printed enclosure
2. Speaker grill opening
3. Microphone opening

Circuit Connections:

After preparing all the components, the next step is connecting the XIAO ESP32S3 Sense to the MAX98357A amplifier and speaker.

1. The MAX98357A communicates with the ESP32-S3 using the I2S audio interface, which allows high-quality digital audio output for voice responses and AI feedback.
2. Connect all the components according to the wiring diagram shown above and test the setup before moving to the PCB stage. Testing the circuit beforehand helps verify that the microphone, speaker, amplifier, and power connections are all functioning correctly.
3. Once the prototype was working properly, the project was moved to a custom PCB design to make the device more compact and suitable for wearable use.

PCB Layout:

To reduce the overall size of the AI Pin:

1. The XIAO ESP32S3 Sense was placed on the top side of the PCB
2. The amplifier and supporting components were placed on the bottom side
3. The speaker was mounted between the PCB and amplifier module in a stacked arrangement

This “sandwich” style design helped minimize the thickness of the device while still leaving enough room for the battery inside the enclosure.

After verifying all the connections and PCB traces, the Gerber files were generated and prepared for manufacturing.

Sponsored by NEXTPCB:

NEXTPCB manufactured the PCB used in this project.

NEXTPCB provides:

1. PCB prototyping
2. PCB assembly (PCBA)
3. Multilayer PCB manufacturing
4. HDI PCB support
5. Flexible and rigid-flex PCB services
6. SMT assembly solutions

Their DFM (Design for Manufacturing) analysis tools help identify PCB design issues before production, reducing errors and improving manufacturing reliability.

For this project, the Gerber files were uploaded, verified, and manufactured to achieve a clean and compact PCB finish suitable for a wearable AI device.

After receiving the manufactured PCB, the assembly process was started by first soldering the SMD components, followed by the larger DIP and through-hole components.

SMD Assembly:

1. First, solder paste was applied to the PCB using a custom stencil ordered along with the PCB.
2. After applying the solder paste, all the SMD components were placed carefully onto the PCB one by one, including the XIAO ESP32S3 Sense module.
3. Once all the components were aligned properly, the PCB was heated on a hot plate to melt the solder paste and complete the soldering process.

DIP Assembly:

1. After completing the SMD assembly, the larger through-hole components were soldered onto the PCB.
2. The speaker was first attached directly onto the PCB. Then, the MAX98357A amplifier module was mounted above the speaker and soldered into place.
3. This stacked arrangement helped reduce the overall size of the device while saving enough internal space for the battery.
4. After completing all the soldering and connections, the hardware assembly was finished and ready for firmware flashing and enclosure installation.

This project uses the ESP Flash Download Tool to upload all the required binary files to the ESP32-S3.

1. Download the Flash Tool:

First, download the ESP Flash Download Tool for Windows.

This tool will be used to upload the firmware files to the XIAO ESP32S3 Sense.

Note: The flashing tool used in this project is only available for Windows.

After downloading, extract the ZIP file and open the tool.

2. Open the Flash Tool:

When the software opens, you will see the chip selection screen.

Configure the settings as follows:

1. Chip Type: ESP32-S3
2. WorkMode: Develop
3. LoadMode: UART

After selecting the correct options, press OK.

3. Download the Firmware Files:

Next, download the GitHub repository containing all the required .bin firmware files for this project.

The repository includes:

1. bootloader.bin
2. xiao.bin
3. partition-table.bin
4. ota-data-initial.bin
5. generated-assets.bin

Extract the downloaded files before continuing.

4. Select the Binary Files:

Inside the Flash Download Tool, select all the required .bin files and enter their corresponding flash addresses.

1. bootloader.bin - 0x0
2. xiaozhi.bin - 0x20000
3. partition-table.bin - 0x8000
4. ota-data-initial.bin - 0xd000
5. generated-assets.bin - 0x600000

Make sure all the checkboxes beside the selected files are enabled.

Also select the correct COM port for your XIAO ESP32S3 Sense board.

5. Erase Existing Firmware:

Before uploading the new firmware, press the Erase button to remove any existing firmware already stored on the XIAO-ESP32-S3.

Wait for the erase process to finish completely before continuing.

6. Upload the Firmware:

After the erase process is completed, press the Start button to begin flashing the firmware.

The tool will now upload all the binary files to the board.

This process may take a few minutes.

7. Finish and Reset:

Once the upload process is completed successfully, press the Reset button on the XIAO ESP32S3 Sense.

If everything was flashed correctly, the device should boot successfully and enter Wi-Fi configuration mode.

After flashing the firmware successfully, the next step is configuring the AI Pin and connecting it to Wi-Fi and the Xiaozhi platform.

Wi-Fi Configuration:

1. Enter Configuration Mode:

Power on the device after flashing the firmware.

After booting, the AI Pin will automatically enter Wi-Fi configuration mode and announce:

“Enter the Wi-Fi configuration mode.”

2. Connect to the Device Hotspot:

Using your phone or laptop, connect to the Wi-Fi network created by the device.

The hotspot name will appear as:

1. xiaozhi

3. Open the Configuration Portal:

After connecting to the hotspot, a configuration page should open automatically.

If the page does not open automatically, manually enter the following address into your browser:

192.168.4.1

4. Enter Wi-Fi Credentials:

Inside the configuration portal:

1. Enter your Wi-Fi name (SSID)
2. Enter your Wi-Fi password

After entering the credentials, press the Connect button.

The device will restart automatically and connect to the internet.

Xiaozhi Platform Setup:

After connecting to Wi-Fi, the AI Pin will communicate with the Xiaozhi server.

1. Listen for the Verification Code:

After restarting, the device will announce a numeric verification code through the speaker.

Example:

“6-7-3-8”

Keep this code ready for the next step.

2. Login to Xiaozhi Platform:

Open the Xiaozhi platform in your browser:

xiaozhi.me

Go to the Console section and log into your account.

3. Add the Device:

Inside the dashboard:

1. Click Add Device
2. Enter the verification code announced by the AI Pin

Once verified, the device will be linked to your account.

Configure the AI Assistant:

After adding the device, you can customize the personality and behavior of your AI assistant.

Available Customizations:

1. Assistant name
2. Language
3. Personality
4. Role description
5. Speaking style

For example, you can create:

1. A Jarvis-style assistant
2. A smart home assistant
3. A technical helper
4. A casual voice companion

You can also use the AI Optimize option to automatically improve and expand your character description.

Note: Keep the final character description under 2000 characters.

After completing the configuration, press Save.

Final Step:

Finally, press the Reset button on the XIAO ESP32S3 Sense.

This allows the device to reconnect and sync the latest settings from the server.

Once restarted, your DIY AI Pin is fully configured and ready to use.

To give the AI Pin a cleaner and more professional appearance, a custom 3D-printed enclosure was designed for the project.

Design and Printing:

1. The enclosure was designed to fit all the internal components in a compact form factor while still keeping access to the speaker, microphone, display, and camera module.
2. The case was 3D printed in black, and a separate white camera ring was added around the camera area to improve the overall look of the device.

Final Assembly:

1. After printing the enclosure, all the project components were carefully assembled inside the case and secured using screws.
2. The compact PCB layout helped fit all the hardware properly inside the enclosure while keeping the AI Pin lightweight and wearable.

Magnetic Mounting System:

1. To make the device easy to wear and attach to clothing, a magnetic cap was added to the back side of the enclosure.
2. This magnetic mounting mechanism allows the AI Pin to attach securely to shirts or jackets without requiring clips or straps, making the final design cleaner and more convenient for daily use.

After completing the hardware assembly and firmware configuration, the final step was testing the AI Pin in real-world scenarios.

The device was tested with different voice and vision-based tasks to demonstrate its functionality and AI capabilities.

Object Recognition:

One of the first tests was object recognition using the onboard camera.

1. The AI Pin was used to identify a camera lens held in front of the device. The assistant successfully recognized the object and responded with information about it while also understanding parts of the surrounding environment.
2. This demonstrated the vision capabilities of the AI Pin and its ability to process visual input in real time.

General Knowledge Responses:

The assistant was also tested with general knowledge questions.

1. In one test, the AI Pin was asked to identify a person shown inside a photo frame. The assistant correctly identified the founder of Raspberry Pi and provided information about Eben Upton.
2. This showed the device’s ability to combine voice interaction with AI-generated responses.

Smart Home and IoT Control:

The AI Pin was also connected with Home Assistant to test smart home functionality.

Using simple voice commands, the assistant was able to:

1. Turn the studio lab lights ON
2. Turn the studio lab lights OFF

This demonstrated how the AI Pin can interact with IoT devices and control smart home equipment using voice commands.

Final Result:

After testing, the DIY AI Pin was fully functional as a wearable voice and vision assistant capable of:

1. Voice interaction
2. Object recognition
3. AI-powered responses
4. Smart home control
5. Real-time assistance

The final build successfully combined compact hardware, wireless connectivity, AI capabilities, and wearable design into a single portable device.

In this project, a compact DIY AI Pin was built using the XIAO ESP32S3 Sense board, combining AI, IoT, and embedded systems into a wearable device.

The AI Pin can recognize objects, respond to voice commands, answer questions, and even control smart home devices using Wi-Fi and AI-based interaction.

With a custom PCB, 3D-printed enclosure, and fully open-source firmware, this project demonstrates how powerful wearable AI devices can be built using affordable hardware and beginner-friendly tools.