Magic Wand with Machine Learning using ML5JS and BLE API
The project uses an Arduino Nano IoT 33 that records real time motion from the IMU (inertial measurement unti) on the Arduino which is transmitted over BLE (bluetooth low energy). A nearby computer running a website in the Google Chrome browser with the experimental BLE API enabled, is able to read the movements of the wand attached to the Arduino. The website uses the popular ML5JS library to create a neural network which can be trained to learn spells.
IMU X,Y,Z measurements are captured at 100Hz, batched into sets of 10 messages to be sent over Bluetooth at 10Hz and below the 251 byte limitation of BLE.
HTML5, P5JS and ML5JS are used to receive the wand motions, these are added to a ring buffer capable of holding 2 seconds of motion - which is estimated size to record the movement for each spell. The ring buffer is constantly updated with the wand's motion, which is then sent to the neural network for classification.
Classification takes around 5ms, with the confidence levels displayed on the page in the P5 canvas. The last spell cast is shown, which has a confidence of more than 0.8 (80%).
Considering the training sets are only 10 repetitions of each of the two spells "Circle" and "Zoro" and only 20 sets of the wand resting and being held. The accuracy seems remarkably high with only a small amount of training data for an object moving around in free space.
I still have much to learn, as this was a time-boxed project of 30 days.
I was so exicted by the end of the project I had to make a video of the project to show my friends and the wider world!
More information about the Daniel Shiffman and his Coding Train videos I l watched to learn all this can be found on my other page here