Tips on how to use ROS 2 shared reminiscence in snaps

In case you already tried to bundle ROS 2 Cunning purposes into snaps, you may need encountered the next error relating to shared reminiscence:

[RTPS_MSG_OUT Error] Area title: fastrtps -> Perform compute_per_allocation_extra_size
[RTPS_TRANSPORT_SHM Error] Did not create section bed1660b134d4b19: Permission denied -> Perform compute_per_allocation_extra_size
[RTPS_MSG_OUT Error] Permission denied -> Perform init

This log is stating that FastDDS (previously often known as FastRTPS) couldn’t create a file for the shared reminiscence mechanism attributable to denied permission. Fortuitously, FastDDS is wise sufficient to fallback to a non-shared memory-based possibility, permitting your program to run simply fantastic.

On this submit, we are going to evaluate the shared reminiscence mechanism in ROS 2, clarify why this error is going on in snaps and suggest totally different options to sort out it in strictly confined snaps. We due to this fact assume that you’re aware of each ROS 2 and snaps. If not, take a look at both (or each) the ROS 2 documentation and the snap touchdown web page.

What’s ROS 2 shared reminiscence

Shared reminiscence is a quick interprocess communication mechanism, permitting a number of processes to entry the shared reminiscence space like common working reminiscence. By making a reminiscence section accessible to a number of processes, these processes can learn/write to this reminiscence with the assistance of a synchronization mechanism. In FastDDS, shared reminiscence normally has a  decrease latency and larger throughput than socket communication attributable to utilizing a smaller-packet based mostly mechanism and environment friendly copy methods.

On Linux, shared reminiscence segments are created below the /dev/shm path. On this listing, one normally finds the reminiscence section recordsdata in addition to synchronization associated recordsdata (locks, mutexes, and so forth.).  Right here is an instance of what ROS 2 Cunning is creating in /dev/shm.

-rw-r--r--  1 guillaume guillaume 545K mai   11 09:40 fastrtps_1ab64bba8500211c
-rw-rw-r--  1 guillaume guillaume    0 mai   11 09:40 fastrtps_1ab64bba8500211c_el
-rw-r--r--  1 guillaume guillaume  33K mai   11 09:40 fastrtps_port7424
-rw-rw-r--  1 guillaume guillaume    0 mai   11 09:40 fastrtps_port7424_el
-rw-r--r--  1 guillaume guillaume  33K mai   11 09:40 fastrtps_port7425
-rw-rw-r--  1 guillaume guillaume    0 mai   11 09:40 fastrtps_port7425_el
-rw-r--r--  1 guillaume guillaume   32 mai   11 09:40 sem.fastrtps_port7424_mutex
-rw-r--r--  1 guillaume guillaume   32 mai   11 09:40 sem.fastrtps_port7425_mutex

The recordsdata with ‘_el’ suffix are locks, recordsdata with the ‘sem.’ prefix are named mutexes, fastrtps_port prefixed recordsdata are ports and the remainder are precise shared reminiscence recordsdata. The named mutexes’s creation will generate some non permanent recordsdata (with the format /dev/shm/XXXXXX ).

Word that FastDDS is utilizing enhance::interprocess as its backend for the shared reminiscence mechanism.

ROS 2 communication can profit from the shared reminiscence mechanism when nodes are on the identical machine. Allow us to discover how this works.

How is ROS 2 Cunning utilizing FastDDS shared reminiscence

FastDDS implements totally different communication mechanisms UDP, TCP and so forth, but additionally the shared reminiscence, enabled by default. Shared reminiscence is used when out there and from the identical host. That means that utilizing shared reminiscence gained’t stop you from monitoring the information from one other machine. It should fall again to the common messaging mechanism. 

An important recordsdata within the shared reminiscence transport below /dev/shm are fastrtps_port* and fastrtps_* recordsdata.

Fastrtps_* incorporates the block of shared reminiscence (section) accessible by a number of processes. Each DomainParticipant configured with shared reminiscence creates such a file on writer/subscriber creation. The info are written to this file and presumably learn by every other area members.

To pay attention to these recordsdata and what knowledge they include, FastDDS is utilizing ports (fastrtps_port* recordsdata). The primary port created is used for participant’s discovery. Each participant is declaring on this port its existence and what port it’s listening to. Any participant would possibly write a knowledge descriptor to every other participant port declaring what knowledge is out there on what shared reminiscence section.

Yow will discover within the FastDDS documentation an instance situation of area members exchanging knowledge by shared reminiscence.

The remaining recordsdata *_el and sem.* are respectively lock recordsdata (to keep away from section and ports zombie recordsdata) and named mutexes for interprocess port entry synchronization.

ROS 2 shared reminiscence in a strictly confined snap

Now that we’ve got reviewed the problem and the technical bits behind it, allow us to talk about options. By default, when strictly confined, snaps don’t have entry to the /dev/shm path on the host machine however solely to a predefined subdirectory particular to every snap following the sample: /dev/shm/snap.SNAP_NAME. The FastDDS shared reminiscence listing of ROS 2 Cunning will not be configurable as of the time of writing. We can’t power it to make use of the shared reminiscence subdirectory created for the snap. 

We are able to discover mentioned options with examples on GitHub.

Public shared reminiscence interface for ROS 2

Snap is offering an interface referred to as shared-memory.  This interface permits totally different snaps to have entry (learn and/or write) to a specified path. That is meant to share sources in /dev/shm throughout snaps. To take action, we’ve got to declare each a slot in addition to plug. Earlier than working the snap, we should manually join these slot & plug, presumably connecting a number of different snaps to the identical slot.

The slot is the one defining the shared reminiscence paths to be accessed. With ROS 2 Cunning, we really need entry to every part throughout the /dev/shm listing (as a result of semaphore non permanent recordsdata). An instance of the slot and plug is as follows:

slots:
  shmem-slot:
    interface: shared-memory
    write: ['*'] # paths are relative to /dev/shm
    personal: false 
plugs:
  shmem-plug:
    interface: shared-memory
    shared-memory: shmem-slot
    personal: false

Then each, the plug and the slot are added to the apps:

apps:
  my-ros-2-app:
    [...]
    plugs: [network, network-bind, shmem-plug]
    slots: [shmem-slot] 

As soon as the snap is constructed and put in, we are able to join the shared reminiscence with the command

sudo snap join my_snap_name:shmem-plug my_snap_name:shmem-slot

Extra snaps will merely should create their very own plug and join it to the exact same slot.

This answer has an vital overhead, since one should outline slots and plugs, presumably throughout a number of snaps. However is the de-facto technique to allow the usage of the shared reminiscence characteristic in strictly confined snaps. Word that snapd 2.56.2 (or above) is critical.

Non-public shared reminiscence interface

Snap is offering one other interface referred to as personal shared reminiscence that vastly simplifies shared reminiscence assist with snap. The personal shared reminiscence is a subset of the shared reminiscence interface. With out modifying your software program, all calls to /dev/shm are going to be certain to /dev/shm/snap.SNAP_NAME mechanically.

In a ROS 2 Cunning context, including the personal shared reminiscence permits the shared reminiscence to work inside a given snap, therefore benefiting from  higher efficiency.

As talked about earlier, the shared reminiscence is used when purposes are working on the identical host. When utilizing the personal shared reminiscence, all of the recordsdata we detailed earlier are mechanically positioned in a path particular to a given snap. Nonetheless, FastDDS doesn’t learn about such a nook case and due to this fact proceeds with establishing a neighborhood communication utilizing the shared reminiscence mechanism. Little does it know that this won’t work. Every little thing is ok throughout the snap, however the software is mute to the surface. ROS 2 purposes working on the host, however outdoors the snap, will be unable to see the matters from the snap; not to mention to  subscribe to them.

Equally, two native customers on the identical host gained’t have write entry to one another’s shared reminiscence recordsdata attributable to permissions. Therefore, one consumer can’t subscribe to a different consumer’s subject by shared reminiscence. Comparable causes, identical outcome.

This being mentioned, you will need to notice that for those who entry these matters revealed from the snap from one other laptop,  it’ll work seamlessly as the information are shared through UDP.

Disabling shared reminiscence for ROS 2

Final however not least, the massive hammer. FastDDS gives two choices to completely disable the shared reminiscence characteristic; both at compile time or at run time. We’re detailing each choices hereafter.

At compile-time

FastDDS gives an choice to compile with out the shared reminiscence characteristic by merely specifying a CMake variable: -DSHM_TRANSPORT_DEFAULT=OFF. With this, no shared reminiscence nor any related recordsdata – ciao the error message. After all, the principle downside of this strategy is that we’ve got to recompile FastDDS with each snap.

Disabling shared reminiscence at run-time

FastDDS additionally permits for offering a configuration XML file at runtime with a view to customise a number of elements of the middleware. Such a side is to power the transport to make use of UDPv4. The XML profile is handed by an surroundings variable: 

FASTRTPS_DEFAULT_PROFILES_FILE: myProfileLocation/fastdds_no_shared_memory.xml

That is a lot simpler to arrange and to vary in subsequent releases of a snap.

Abstract

I hope that studying this text helps you perceive a bit extra concerning the ROS 2 Cunning shared reminiscence characteristic and how one can deal with it with snaps. Though the shared reminiscence mechanism has clear advantages, it additionally comes with constraints. Now we have seen that file permissions and even containerisation may cause the shared reminiscence to not work as anticipated. Understanding the shared reminiscence implementation from FastDDS in ROS 2 Cunning helped us to determine options to make use of it in snaps along with their limitations. Typically instances, the only answer is to bundle the complete stack right into a single snap utilizing the personal shared reminiscence, or to easily disable it. 

When you have every other suggestions or concepts relating to ROS 2 Cunning and shared reminiscence, please be part of our discussion board and tell us what you assume. Moreover, take a look on the snap documentation if you wish to study extra about snaps for robotics purposes.