Robotic drawing communication protocol: a framework for building a semantic drawn language for robotic fabrication

Robots are usually operated through text-based inputs made on an external computer or through an associated Human Machine Interface (HMI). This requires skill and expert knowledge to take full advantage of the robot as machine, tool, and extension of the human operator, thus limiting applications for users that hold manual skills but not machine knowledge. Consequently, this research aims to identify processes that allow non-specialist to operator a robot with a similar ease as a specialist.
This paper presents research into minimizing (or fully negating) text-based programming for robotic fabrication, thereby opening a potential for adopting robotic fabrication by users with broader level of skills. This can be achieved by introducing a process for non-specialists to use a semantic drawn language, whereby manual instructions are drawn on a workpiece before being robotically processed. The language can be extended by the operator through interaction with a Machine Learning (ML) system operated on an HMI, which parses the language and informs the robot what to do.
The paper discusses further research into a previously developed tablet inter-face framework that manages this process; and specifically details the process of adding ML functionalities that can continuously improve the framework. It de-scribes the development process of a data gathering method; provides an over-view use cases for classification results and choice of training system; and dis-cusses results and limitations, with discussion of future work.