Other products

ROBOTCORE Perception

Speed up your ROS robotics perception pipelines

ROBOTCORE Perception is an optimized robotic perception stack that leverages hardware acceleration to provide a speedup in your perception computations. API-compatible with the ROS 2 perception stack, ROBOTCORE Perception delivers high performance, real-time and reliability to your robots' perception.

Get ROBOTCORE® Perception Perception consulting
ROBOTCORE PERCEPTION

Accelerated robotics perception

Data obtained in a robot from its sensors like cameras and lidar is typically fed into the perception turning into something useful for decision making and planning physical actions. Perception helps sense the static and dynamic objects, and build a reliable and detailed representation of the robot's environment using computer vision and machine learning techniques. The perception layer in a robot is thereby responsible for object detection, segmentation and tracking. Traditionally, a perception pipeline starts with image pre-processing, followed by a region of interest detector and then a classifier that outputs detected objects.

ROBOTCORE Perception is a hardware accelerated perception stack served as either source code or IP that allows your perception pipelines to reduce their runtime latency and increase throughput. Delivering a perception speedup, it's API-compatible with the ROS 2 perception stack simplifying its integration in existing robots, so that you don't spend time reinventing the wheel and re-developing what already works.

Benchmarks

Default ROS
perception

0.7 Hz/image for a 3-Node perception pipeline in embedded

A simple Rectify-Resize-Harris ROS perception pipeline is demonstrated which performs pre-processing (Rectify and Resize) and region of interest detection (Harris corners detector). This simple pipeline consisting of only 3 ROS Nodes takes on average 1266.32 milliseconds to execute for a single pass, which leads to less than 1 Hz when running in an idle embedded Quad-core arm Cortex-A53.

ROBOTCORE
Perception

4.5x speedup in simple ROS perception pipelines

Faster robots require faster perception. ROBOTCORE Perception delivers exactly this upgrading perception pipelines with a 4.5x speedup, impacting pipeline runtime and throughput in embedded. More complex perception graphs can further leverage acceleration, achieving 10x speedups and above.

Towards more
capable robots

Moving faster (or with more dexterity) requires faster perception computations. Atlas below uses perception to identify, navigate and jump around obstacles. Edge perception is key to navigate under environments that change, which are most in the human world. Atlas leverages a time-of-flight depth camera to generate point clouds of the environment at 15 Hz. The point cloud is a large and dense collection of range measurements which needs to be processed at the edge (directly in the robot) for smooth behaviors, reducing any latency from the sensors and all the way into the actuators. By leveraging hardware acceleration, Atlas’ perception extracts surfaces from this point cloud which are then used to plan actions in the order of tenths of milliseconds. All happening on the edge.

ROBOTCORE Perception delivers faster perception capabilities to ROS computational graphs. So that your robots can move faster and better.

Co-developed
with the best

ROBOTCORE Perception is the result of joint research and co-development with computer system architects from Harvard University.

Read the paper

Developer-ready documentation and support

ROBOTCORE Perception is served by seasoned ROS developers for ROS development. It includes documentation, examples, reference designs and the possibility of various levels of support.

Ask about support levels

Benchmarks

(plots are interactive)

ROS 2 PERCEPTION NODES

(Measurements discard ROS 2 message-passing infrastructure overhead and host-device (GPU or FPGA) data transfer overheads)

Resize - speedup

2.61x

Resize - kernel runtime latency (ms) (ROBOTCORE Perception running on an AMD KV260, NVIDIA Isaac ROS running in a Jetson Nano 2GB. Measurements present the kernel runtime in milliseconds (ms) and discard ROS 2 message-passing infrastructure overhead and host-device (GPU or FPGA) data transfer overhead)

Resize - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Rectify - speedup

7.34x

Rectify - kernel runtime latency (ms)

Rectify - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Harris - speedup

30.27x

Harris - kernel runtime latency (ms)

Harris - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Histogram of Oriented Gradients - speedup

509.52x

Histogram of Oriented Gradients - kernel runtime latency (ms)

Histogram of Oriented Gradients - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Canny Edge Tracing - speedup

3.26x

Canny Edge Tracing - kernel runtime latency (ms)

Canny Edge Tracing - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Fast Corner Detection - speedup

8.43x

Fast Corner Detection - kernel runtime latency (ms)

Fast Corner Detection - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Gaussian Difference - speedup

11.94x

Gaussian Difference - kernel runtime latency (ms)

Gaussian Difference - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Bilateral Filter - speedup

9.33x

Bilateral Filter - kernel runtime latency (ms)

Bilateral Filter - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

Stereo LBM - speedup

5.19x

Stereo LBM - kernel runtime latency (ms)

Stereo LBM - % resource consumption (LUT, FF, DSP, BRAM)
(Considering an AMD KV260 board for resource estimation and an image with 4K resolution as input. Additional details available in the ROBOTCORE Perception documentation.)

ROS 2 PERCEPTION GRAPHS

2-Node pre-processing perception graph latency (ms)
(Simple graph with 2 Nodes (Rectify-Resize) demonstrating perception pre-processing with the image_pipeline ROS 2 package. AMD's KV260 and NVIDIA's Jetson Nano 2GB boards are used for benchmarking, the former featuring a Quad-core arm Cortex-A53 and the latter a Quad-core arm Cortex-A57. Source code used for the benchmark is available at the perception_2nodes ROS 2 package)

2-Node pre-processing perception graph performance-per-watt (Hz/W)

Graph speedup - 2-Node pre-processing perception graph latency

3.6x

Performance-per-watt improvement - 2-Node pre-processing perception graph

6x

3-Node pre-processing and region of interest detector perception graph latency (ms)
(3-Nodes graph (Rectify-Resize-Harris) demonstrating perception pre-processing and region of interest detection with the image_pipeline ROS 2 package. AMD's KV260 featuring a Quad-core arm Cortex-A53 is used for benchmarking. Source code used for the benchmark available at the perception_3nodes ROS 2 package)

Graph speedup - 3-Node pre-processing and region of interest detector perception graph

4.5x

Do you have any questions?

Get in touch with our team.

Let's talk