r4robot

2021

Founded a research studio to develop tools and interactions to train intelligent robots. Exploring concepts in robotics and machine learning.

Visit r4robot.com

Concept

A robotic arm learns to recognize new objects and tasks through observation.

 

Motivation

Affordable robotic automation for small businesses and manufacturers of seasonal or highly variable products. Create a system that enables a robotic arm to be trained on new tasks within seconds.

Working Principle

The robot learns to recognize new objects by ‘playing’ with them, building up a visual dataset to train a neural network for object detection. Tasks are learned by tracking hand movements and actions with a custom stereocamera.

Prototypes

Initial prototypes built with Arduino Braccio and Arduino Leonardo. Later prototypes built with uFactory xArm 5 Lite and NVIDIA Jetson Xavier NX.

Custom-built components:

  • Rebellum software product.

  • Depth sensing algorithm.

  • Algorithms for collecting images, labeling images with bounding boxes, transfer learning, and actuating robot arm.

  • Inverse kinematics algorithm.

  • Stereocamera with two low-cost webcams.

  • 3D printed gimbal.

  • Arduino library to control hobbyist robot motors (non-blocking).

  • Serial data flow between computer and Arduino with RTS/CTS flow control.

  • Motor angle calibration.

 

Off-the-shelf components:

  • MediaPipe Hands solution to detect human hand and fingers.

  • Logitech C615 webcams.

  • USB to TTL serial cable.

  • Serial communication libraries.