r4robot
2021
Founded a research studio to develop tools and interactions to train intelligent robots. Exploring concepts in robotics and machine learning.
Visit r4robot.com
Concept
A robotic arm learns to recognize new objects and tasks through observation.
Motivation
Affordable robotic automation for small businesses and manufacturers of seasonal or highly variable products. Create a system that enables a robotic arm to be trained on new tasks within seconds.
Working Principle
The robot learns to recognize new objects by ‘playing’ with them, building up a visual dataset to train a neural network for object detection. Tasks are learned by tracking hand movements and actions with a custom stereocamera.
Prototypes
Initial prototypes built with Arduino Braccio and Arduino Leonardo. Later prototypes built with uFactory xArm 5 Lite and NVIDIA Jetson Xavier NX.
Custom-built components:
Rebellum software product.
Depth sensing algorithm.
Algorithms for collecting images, labeling images with bounding boxes, transfer learning, and actuating robot arm.
Inverse kinematics algorithm.
Stereocamera with two low-cost webcams.
3D printed gimbal.
Arduino library to control hobbyist robot motors (non-blocking).
Serial data flow between computer and Arduino with RTS/CTS flow control.
Motor angle calibration.
Off-the-shelf components:
MediaPipe Hands solution to detect human hand and fingers.
Logitech C615 webcams.
USB to TTL serial cable.
Serial communication libraries.