Hands off shared memory!

Pitfalls

• Shared mutable state. This pitfall is intentional. After all, sharing is the point of using shared memory. Nonetheless, ensuring data races do not happen is hard to get right, especially because you cannot use any of the standard library’s threading support. Those facilities are designed with threads in mind and do not work for inter-process coordination.

• Object ownership. In a way all objects in the shared memory are dynamically allocated. They must be treated as resources, which brings up the question of ownership. For unique ownership std::unique_ptr with a custom deleter should work. But shared ownership requires a custom implementation. std::shared_ptr won’t do because it only has a thread safe use count, not a process safe one.

• Object lifetimes. Creating and releasing the shared memory region is separated from constructing and destroying objects in there, which means you are in the land of placement new and explicit destructor calls.

  But that’s not all. Imagine a server process initializing the shared memory block and placement newing objects into it. Then a client process attaches to that shared memory block. What it sees is just raw memory. Even if it knows exactly where specific objects are located, simply reinterpret casting those addresses invokes UB.

• No pointers to the outside. Such a pointer destination would not be accessible from all but one of the processes, ruling out at least all types that allocate memory. No std::vector, no std::string, etc.

• No pointers inside, a.k.a. differing base addresses. You get no guarantee that the shared memory block is mapped at the same base address in all involved processes. If you want to have a pointer in the shared mem pointing to another location in the shared mem, tough luck! The destination is almost certain to live at a different address in each involved process.

  Biggest problem here: virtual class hierarchies. Almost certainly your compiler implements virtual functions with vtables, which contain pointers.

• Full ABI compatibility required. Using the identical object in multiple processes requires all processes to have the exact same understanding of what the bits of that object mean.

  Not screwing up ABI compatibility is hard enough as it is. Shared memory makes it worse because without pointers one of the main ABI guaranteeing techniques – pimpl – becomes more difficult to implement.

Mitigation

• Allowing only certain kinds of types in the shared memory can go a long way to mitigate many of the pitfalls. The more the shared objects are “dumb C-style structs” the better. More formally, go for standard layout types without any pointers. Obvious downside: A large part of the type system’s power and flexibility becomes unusable.

• On POSIX systems there is a way around the base address problem. When you use fork() to copy your process and continue running the copy – i.e. you do not call exec() –, the base addresses are the same as a consequence of how forking works. Downsides:

  • Building all the functionality of all the involved processes into a single program is a severe architecture restriction.
  • You lose platforms that don’t provide either fork() or an equivalent mechanism. In practice you lose Windows.

• If the fork() approach is viable, a custom allocator could re-enable a lot of the standard library types. However, at that point you’re essentially building your very own memory management from scratch.

• The object lifetime problems with reinterpreting raw shared memory as objects are tricky to solve. In C++23 we’ll have std::start_lifetime_as to address such situations directly. Until then std::launder can help. Probably. The wording in the standard sounds promising, but I’m not entirely sure if it really covers cross-process situations, too. Depending on the exact shared types std::bit_cast or the classic memcpy() trick might be viable, too.

  I’m confident you could find a definite solution for this problem with a bit more research. But I’ll wait with putting in the effort until I’ll have to use shared memory in earnest.

Bottom line

In terms of the likelihood to get it wrong, shared memory plays in the same league as manual memory management. It’s not a question of if, just of how and when.

In some niche situations it might be the right tool to solve a concrete problem, probably a performance related one. If you do encounter such a situation, don’t screw around. Keep the kinds of objects in the shared memory as simple as possible. Write abstractions. Make sure shared memory details and domain logic never ever come even close to each other.

Simply letting some arbitrary C++ objects live in a region of shared memory without heavy-duty safeguards will not end well.