Project Silent Willow was originally a side/passion project I worked on in the last two years of middle school but has developed to be my largest project to-date, integrating hardware design engineering and software/web development!
As an overview, it's a vehicle that's capable of moving around a room autonomously using just LiDAR and image recognition (both programs run on two Jetson nanos). As a mechanum wheeled vehicle, it's also able to travel different directions without having to pivot around a point.
It also happens to be my first ever project integrating AI in any capacity (as well as getting accredited for it). As explained in the video below, the vehicle currently only relies on image recognition and "reference QR codes" to make decisions on how to move the vehicle. In the future, I'm planning to integrate the LiDAR unit for general room navigation.
I'm currently working on developing apps for Windows and Android to control the vehicle. I initially used MIT App Inventor for basic control but I wanted more customization with for example how the app is formatted. That's why I've switched to using Android Studio and Visual Studio for further app development.
Technology
I initially went with an Arduino MEGA 2560 (as it had the most GPIOs available) but due to the space restrictions, I'm currently working with my custom microcontroller (tau-w) which includes built-in bluetooth. Also, because the ESP32S3 module I'm using includes bluetooth, I could remove the need to use an HC-05 (which was my initial module for communicating to the vehicle).
For driving the motors, I initially went with 4 DRV8825 modules but they were quite inefficient, so I've switched to using TMC2209 drivers (which during initial tests were considerably more efficient and quiet).
For anything related to image recognition and LiDAR, I opted with Nvidia Jetson Nanos. These were cheap enough for the project and had enough computational power to carry out the tasks.
Links
Check out the following links for more info and specifics about this project:
Most of the issues I encountered (and am still encountering) are pretty small so they won't be included here but these two issues caused major setbacks for the project:
One of the largest yet smallest issues (I know, ironic) was the baud rate I initially set for Bluetooth communications with the vehicle. This is because I initially set the communication baud rate to 115200 for the HC-05 module but found out it operates transmissions at 9600. This ended up setting the project back a few months as I could not find that issue.
Another issue that was software related was when I initially ran the image recognition program. The issue was that when classifying what was seen in the camera, my program kept giving false positives and negatives. This meant the vehicle started moving when it was not meant to and vice versa. To fix this I simply uploaded more pictures of the "reference QR codes" for training and included pictures with different lighting conditions.
