memfd_create(2)

For 4 months now we’ve been hacking on a new syscall for the linux-kernel, called memfd_create. The intention is to provide an easy way to get a file-descriptor for anonymous memory, without requiring a local tmpfs mount-point. The syscall takes 2 parameters, a name and a bunch of flags (which I will not discuss here):

int memfd_create(const char *name, unsigned int flags);

If successful, a new file-descriptor pointing to a freshly allocated memory-backed file is returned. That file is a regular file in a kernel-internal filesystem. Therefore, most filesystem operations are supported, including:

  • ftruncate(2) to change the file size
  • read(2), write(2) and all its derivatives to inspect/modify file contents
  • mmap(2) to get a direct memory-mapping
  • dup(2) to duplicate file-descriptors

Theoretically, you could achieve similar behavior without introducing new syscalls, like this:

int fd = open("/tmp/random_file_name", O_RDWR | O_CREAT | O_EXCL, S_IRWXU);
unlink("/tmp/random_file_name");

or this

int fd = shm_open("/random_file_name", O_RDWR | O_CREAT | O_EXCL, S_IRWXU);
shm_unlink("/random_file_name");

or this

int fd = open("/tmp", O_RDWR | O_TMPFILE | O_EXCL, S_IRWXU);

Therefore, the most important question is why the hell do we need a third way?

Two crucial differences are:

  • memfd_create does not require a local mount-point. It can create objects that are not associated with any filesystem and can never be linked into a filesystem. The backing memory is anonymous memory as if malloc(3) had returned a file-descriptor instead of a pointer. Note that even shm_open(3) requires /dev/shm to be a tmpfs-mount. Furthermore, the backing-memory is accounted to the process that owns the file and is not subject to mount-quotas.
  • There are no name-clashes and no global registry. You can create multiple files with the same name and they will all be separate, independent files. Therefore, the name is purely for debugging purposes so it can be detected in task-dumps or the like.

To be honest, the code required for memfd_create is 100 lines. It didn’t take us 2 months to write these, but instead we added one more feature to memfd_create called Sealing:

File-Sealing

File-Sealing is used to prevent a specific set of operations on a file. For example, after you wrote data into a file you can seal it against further writes. Any attempt to write to the file will fail with EPERM. Reading will still be possible, though. The crux of this matter is that seals can never be removed, only added. This guarantees that if a specific seal is set, the information that is protected by that seal is immutable until the object is destroyed.

To retrieve the current set of seals on a file, you use fcntl(2):

int seals = fcntl(fd, F_GET_SEALS);

This returns a signed 32bit integer containing the bitmask of currently set seals on fd. Note that seals are per file, not per file-descriptor (nor per file-description). That means, any file-descriptor for the same underlying inode will share the same seals.

To seal a file, you use fcntl(2) again:

int error = fcntl(fd, F_ADD_SEALS, new_seals);

This takes a bitmask of seals in new_seals and adds these to the current set of seals on fd.

The current set of supported seals is:

  • F_SEAL_SEAL: This seal prevents the seal-operation itself. So once F_SEAL_SEAL is set, any attempt to add new seals via F_ADD_SEALS will fail. Files that don’t support sealing are initially sealed with just this flag. Hence, no other seals can ever be set and thus do not have to be enforced.
  • F_SEAL_WRITE: This is the most straightforward seal. It prevents any content modifications once it is set. Any write(2) call will fail and you cannot get any shared, writable mappings for the file, anymore. Unlike the other seals, you can only set this seal if no shared, writable mappings exist at the time of sealing.
  • F_SEAL_SHRINK: Once set, the file cannot be reduced in size. This means, O_TRUNC, ftruncate(), fallocate(FALLOC_FL_PUNCH_HOLE) and friends will be rejected in case they would shrink the file.
  • F_SEAL_GROW: Once set, the file size cannot be increased. Any write(2) beyond file-boundaries, any ftruncate(2) that increases the file size, and any similar operation that grows the file will be rejected.

Instead of discussing the behavior of each seal on its own, the following list shows some examples how they can be used. Note that most seals are enforced somewhere low-level in the kernel, instead of directly in the syscall handlers. Therefore, side effects of syscalls I didn’t cover here are still accounted for and the syscalls will fail if they violate any seals.

  • IPC: Imagine you want to pass data between two processes that do not trust each other. That is, there is no hierarchy at all between them and they operate on the same level. The easiest way to achieve this is a pipe, obviously. However, to allow zero-copy (assuming splice(2) is not possible) the processes might decide to use memfd_create to create a shared memory object and pass the file-descriptor to the remote process. Now zero-copy only makes sense if the receiver can parse the data in-line. However, this is not possible in zero-trust scenarios as the source can retain a file-descriptor and modify the contents while the receiver parses it, causing any kinds of failure. But if the receiver requires the object to be sealed with F_SEAL_WRITE | F_SEAL_SHRINK, it can safely mmap(2) the file and parse it inline. No attacker can alter file contents, anymore. Furthermore, this also allows safe mutlicasts of the message and all receivers can parse the same zero-copy file without affecting each other. Obviously, the file can never be modified again and is a one-shot object. But this is inherent to zero-trust scenarios. We did implement a recycle-operation in case you’re the last user of an object. However, that was dropped due to horrible races in the kernel. It might reoccur in the future, though.
  • Graphics-Servers: This is a very specific use-case of IPC and usually there is a one-way trust relationship from clients to servers. However, a server cannot blindly trust its clients. So imagine a client renders its window-contents into memory and passes a file-descriptor to that memory region (maybe using memfd_create) to the server. Similar to the previous scenario, the server cannot mmap(2) that object for read-access as the client might truncate the file simultaneously, causing SIGBUS on the server. A server can protect itself via SIGBUS-handlers, but sealing is a much simpler way. By requiring F_SEAL_SHRINK, the server can be sure, the file will never shrink. At the same time, the client can still grow the object in case it needs bigger buffers for growing windows. Furthermore, writing is still allowed so the object can be re-used for the next frame.

As you might imagine, there are a lot more interesting use-cases. However, note that sealing is currently limited to objects created via memfd_create with the MFD_ALLOW_SEALING flag. This is a precaution to make sure we don’t break existing setups. However, changing seals of a file requires WRITE-access, thus it is rather unlikely that sealing would allow attacks that are not already possible with mandatory POSIX locks or similar. Hence, it is possible that sealing will expand to other areas in case people request it. Further seal-types are also possible.

Current Status

As of June 2014 the patches for memfd_create and sealing have been publicly available for at least 2 months and are considered for merging. linux-3.16 will probably not include it, but linux-3.17 very likely will. Currently, there’s still some issues to be figured out regarding AIO and Direct-IO races. But other than that, we’re good to go.

About these ads

11 thoughts on “memfd_create(2)

  1. Andreas

    Just a quick question:
    Can I poll on a memfd ?
    Esp. can I poll on appearance of a new seal on a memfd ?

    Thanks,
    Andreas

    Reply
      1. Andreas

        I guess that was some blunder of mine. Too much stream-based IPC lately…

        I had thought about using receiver-SEAL_WRITE and sender-POLLLPRI for reception-notification such that the sender could close the fd or similar stuff. But since the fd needs to be passed via some udx there is no need for this anyways I guess… A last-user-event could be nice occasionally I guess, but again, there is a udx available, so nevermind I asked… ;-)

  2. Pritam Baral

    This is cool!

    Just one question:
    I read downstream that this can lead to ashmem-like use-cases. I don’t know enough about shmem to be able to deduce if the reserved memory will be automatically cleared.

    Like, POSIX shm does clear memory after all processes call shm_unlink, but if either process crashes before calling shm_unlink, that named memory remains reserved.

    So, my question is: will memory reserved by ftruncate on a fd from memfd_create be automatically cleared after all processes stop using it, gracefully or otherwise?

    Reply
    1. David Herrmann Post author

      Yes, of course. If all file-descriptors to the file are dropped, the file is even destroyed (as there is no way to access it anymore). There’re not ugly cleanup races like with SHM.

      Reply
  3. jrch2k8Rafael Castillo

    Hi, david couple of questions
    1.) i’m playing with your upstream memfd code and i noticed the same restriction of regular fds exist in memfds, the numbering table exist privately per pid. there is any way to pass this fd to another process without regular methods or KDBUS? (Seals are awesome and open to suggestions btw).

    2.) i noticed that mmap(MAP_SHARED or MAP_PRIVATE) fails when F_SEAL_WRITE is set(the docs seems to point this is right tho), is there any way to mmap a write sealed fd or is not recommended? (open to suggestions)

    Awesome work, many thanks

    Reply
    1. David Herrmann Post author

      Hi, feel free to send me emails. That’s usually easier.
      Anyway, I don’t understand your first question. I mean, the purpose of memfd is to get a file-descriptor for memory regions, so obviously the “regular-methods” to share those are required.

      Regarding 2), I don’t think that’s right. mmap(MAP_PRIVATE) always works on sealed files. What error do you get? Obviously, MAP_SHARED does not work on WRITE-sealed files, but that’s not needed, anyway, as it is read-only.

      Reply
  4. Stas

    Hi David, great work!
    A question: would it be possible to have a similar call
    but that takes a pointer to the existing memory region?
    Currently vmsplice() allows connecting the memory regions
    to pipe, but your approach allows ftruncate() and mmap(),
    which AFAIK is not possible with pipe. Your approach does
    not involve pipe, which is IMHO very good.
    My idea is to make it possible to share any VA regions of
    a running process, either with its own or with another process.
    In the past this was possible by doing mmap() on /proc//mem,
    but then that functionality was disabled forever.
    It will also cheaply solve this problem:
    https://pax.grsecurity.net/docs/vmmirror.txt
    Do you think something like this would be sensible and
    possible with your code?

    Reply
    1. David Herrmann Post author

      I never intended to support such use-cases. A process might have arbitrary hardware registered mapped in their address-space, we really don’t want to pull those pages into a memfd. memfds are about safely sharing data between processes without any trust-relationship. vma-mirroring really doesn’t fit in that model. At least I don’t see how memfds would help you there.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s