Heavy metallic parts are manipulated by machinery and robots with tasks involving soldering and
assembly. The environment is characterized by large robots and slow-communicating large
machinery and equipment, with workers overseeing the slow process. In our work with Grupo Celsa
we accelerated their robotic vision pipelines by more than
62x, down to milliseconds, enabling faster robot manipulation and real-time quality control.
The challenge
The traditional process of soldering and assembling heavy metallic parts is slow and
labor-intensive. These parts often have unstructured surfaces, requiring robots to rely on
advanced perception systems powered by computer vision for precise manipulation. The
challenge lies in speeding up these perception systems to enhance robotic efficiency and
enable real-time quality control.
Identifying bottlenecks
Following a systems architecture approach, we identified the bottlenecks in the perception
pipeline, which included the acquisition, 3D comparison, 3D elaboration, preprocessing,
segmentation, 3D localization, postprocessing, and point selection stages.
Accelerating perception
We accelerated the perception pipeline by reimplementing the vision processing with modern
techniques while leveraging ROBOTCORE® Perception, an optimized
robotic perception stack that makes use of hardware acceleration to provide speedups.
Digital twin-driven co-development
We used digital twins to co-develop the perception system and the robot manipulation tasks in parallel, enabling us to iterate quickly and optimize the system for real-world performance.
Robot eyes
By adopting the perspective of "robot eyes," we harnessed cutting-edge computer vision techniques to enhance robot perception and accelerate manipulation.
Leveraging state-of-the-art robot simulation
We used state-of-the-art robot simulation tools to create digital twins, selecting the most suitable one for each task.
Going beyond, perception robustness
We ensured the robustness of the perception system by testing it in a variety of scenarios and
environments, including different lighting conditions and surface textures.
Result: 62x Faster Robot Perception
As a result of our work, we achieved a more than 62x
speedup in the perception pipeline, enabling faster robot manipulation and
real-time quality control. The digital twin-driven development approach allowed us to speed-up
the development process and iterate across multiple teams of robotic engineers and software
developers.
