Author Archives: David Herrmann

GNOME Westcoast Summit 2014

Two months ago in April ’14 I’ve been in San Francisco to meet with other FOSS developers and discuss current projects. There were several events, including the first GNOME Westcoast Summit and a systemd hackfest at Pantheon. I’ve been working on a lot of stuff lately and it was nice to talk directly to others about it. I wrote in-depth articles (on this blog) for the most interesting stories, but below is a short overview of what I focused on in SF:

  • memfd: My most important project currently is memfd. We fixed several bugs and nailed down the API. It was also nice to get feedback from a lot of different projects about interesting use-cases that we didn’t think of initially. As it turns out, file-sealing is something a lot of people can make great use of.
  • MiracleCast: For about half a year I worked on the first Open-Source implementation of Miracast. It’s still under development and only working Sink-side, but there are plans to make it work Client-side, too. Miracast allows to replace HDMI cables with wireless-solutions. You can connect your monitor, TV or projector via standard wifi to your desktop and use it as mirror or desktop-extension. The monitor is sink-side and MiracleCast can already provide a full Miracast stack for it. However, for the more interesting Source-side (eg., a Gnome-Desktop) I had a lot of interesting discussions with Gnome developers how to integrate it. I have some prototypes running locally, but it will definitely take a lot longer before it works properly. However, the current sink-side implementation has a latency of approx. 50ms and can run 30fps 1080p. This is already pretty impressive and on-par with proprietary solutions.
  • kdbus: The new general-purpose IPC mechanism is already fleshed out, but we spent a lot of time fixing races in the code and doing some general code review. It is a very promising project and all of the criticism I’ve heard so far was rubbish. People tend to rant about moving dbus in the kernel, even though kdbus really has nearly nothing to do with dbus, except that it provides an underlying data-bus infrastructure. Seriously, the helpers used for kernel-mode-setting without including the driver-specific code is already much bigger than kdbus… and in my opinion, kdbus will make dbus a lot more efficient and appealing to new developers.
  • GPU: GPU-switching, offload-GPUs and USB/wifi display-controllers are few of the many new features in the graphics subsystem. They’re mostly unsupported in any user-space, so we decided to change that. It’s all highly technical and the way how it is supposed to work is fairly obvious. Therefore, I will avoid discussing the details here. Lets just say, on-demand and live GPU-switching is something I’m making possible as part of GSoC this summer.
  • User-bus: This topic sounds fairly boring and technical, but it’s not. The underlying question is: What happens if you log in multiple times as the same user on the same system? Currently, a desktop system either rejects multiple logins of the same user or treats them as separate, independent logins. The second approach has the problem that many applications cannot deal with this. Many per-user resources have to be shared (like the home-directory). Firefox, for instance, cannot run multiple times for the same user. However, no-one wants to prevent multiple logins of the same user, as it really is a nice feature. Therefore, we came up with a hybrid aproach which basically boils down to a single session shared across all logins of the same user. So if you login twice, you get the same screen for both logins sharing the same applications. The window-manager can put you on a separate virtual desktop, but the underlying session is basically the same. Now if you do the same across multiple seats, you simply merge both sessions of these seats into a single huge session with the screen mirrored across all assigned monitors. A more in-depth article will follow once the details have been figured out.

A lot of the things I worked on deal with the low-level system and are hardly visible to the average Gnome user. However, without a proper system API, there’s no Gnome and I’m very happy the Gnome Foundation is acknowledging this by sponsoring my trip to SF: Thanks a lot! And hopefully I’ll see you again next year!

memfd_create(2)

For 4 months now we’ve been hacking on a new syscall for the linux-kernel, called memfd_create. The intention is to provide an easy way to get a file-descriptor for anonymous memory, without requiring a local tmpfs mount-point. The syscall takes 2 parameters, a name and a bunch of flags (which I will not discuss here):

int memfd_create(const char *name, unsigned int flags);

If successful, a new file-descriptor pointing to a freshly allocated memory-backed file is returned. That file is a regular file in a kernel-internal filesystem. Therefore, most filesystem operations are supported, including:

  • ftruncate(2) to change the file size
  • read(2), write(2) and all its derivatives to inspect/modify file contents
  • mmap(2) to get a direct memory-mapping
  • dup(2) to duplicate file-descriptors

Theoretically, you could achieve similar behavior without introducing new syscalls, like this:

int fd = open("/tmp/random_file_name", O_RDWR | O_CREAT | O_EXCL, S_IRWXU);
unlink("/tmp/random_file_name");

or this

int fd = shm_open("/random_file_name", O_RDWR | O_CREAT | O_EXCL, S_IRWXU);
shm_unlink("/random_file_name");

or this

int fd = open("/tmp", O_RDWR | O_TMPFILE | O_EXCL, S_IRWXU);

Therefore, the most important question is why the hell do we need a third way?

Two crucial differences are:

  • memfd_create does not require a local mount-point. It can create objects that are not associated with any filesystem and can never be linked into a filesystem. The backing memory is anonymous memory as if malloc(3) had returned a file-descriptor instead of a pointer. Note that even shm_open(3) requires /dev/shm to be a tmpfs-mount. Furthermore, the backing-memory is accounted to the process that owns the file and is not subject to mount-quotas.
  • There are no name-clashes and no global registry. You can create multiple files with the same name and they will all be separate, independent files. Therefore, the name is purely for debugging purposes so it can be detected in task-dumps or the like.

To be honest, the code required for memfd_create is 100 lines. It didn’t take us 2 months to write these, but instead we added one more feature to memfd_create called Sealing:

File-Sealing

File-Sealing is used to prevent a specific set of operations on a file. For example, after you wrote data into a file you can seal it against further writes. Any attempt to write to the file will fail with EPERM. Reading will still be possible, though. The crux of this matter is that seals can never be removed, only added. This guarantees that if a specific seal is set, the information that is protected by that seal is immutable until the object is destroyed.

To retrieve the current set of seals on a file, you use fcntl(2):

int seals = fcntl(fd, F_GET_SEALS);

This returns a signed 32bit integer containing the bitmask of currently set seals on fd. Note that seals are per file, not per file-descriptor (nor per file-description). That means, any file-descriptor for the same underlying inode will share the same seals.

To seal a file, you use fcntl(2) again:

int error = fcntl(fd, F_ADD_SEALS, new_seals);

This takes a bitmask of seals in new_seals and adds these to the current set of seals on fd.

The current set of supported seals is:

  • F_SEAL_SEAL: This seal prevents the seal-operation itself. So once F_SEAL_SEAL is set, any attempt to add new seals via F_ADD_SEALS will fail. Files that don’t support sealing are initially sealed with just this flag. Hence, no other seals can ever be set and thus do not have to be enforced.
  • F_SEAL_WRITE: This is the most straightforward seal. It prevents any content modifications once it is set. Any write(2) call will fail and you cannot get any shared, writable mappings for the file, anymore. Unlike the other seals, you can only set this seal if no shared, writable mappings exist at the time of sealing.
  • F_SEAL_SHRINK: Once set, the file cannot be reduced in size. This means, O_TRUNC, ftruncate(), fallocate(FALLOC_FL_PUNCH_HOLE) and friends will be rejected in case they would shrink the file.
  • F_SEAL_GROW: Once set, the file size cannot be increased. Any write(2) beyond file-boundaries, any ftruncate(2) that increases the file size, and any similar operation that grows the file will be rejected.

Instead of discussing the behavior of each seal on its own, the following list shows some examples how they can be used. Note that most seals are enforced somewhere low-level in the kernel, instead of directly in the syscall handlers. Therefore, side effects of syscalls I didn’t cover here are still accounted for and the syscalls will fail if they violate any seals.

  • IPC: Imagine you want to pass data between two processes that do not trust each other. That is, there is no hierarchy at all between them and they operate on the same level. The easiest way to achieve this is a pipe, obviously. However, to allow zero-copy (assuming splice(2) is not possible) the processes might decide to use memfd_create to create a shared memory object and pass the file-descriptor to the remote process. Now zero-copy only makes sense if the receiver can parse the data in-line. However, this is not possible in zero-trust scenarios as the source can retain a file-descriptor and modify the contents while the receiver parses it, causing any kinds of failure. But if the receiver requires the object to be sealed with F_SEAL_WRITE | F_SEAL_SHRINK, it can safely mmap(2) the file and parse it inline. No attacker can alter file contents, anymore. Furthermore, this also allows safe mutlicasts of the message and all receivers can parse the same zero-copy file without affecting each other. Obviously, the file can never be modified again and is a one-shot object. But this is inherent to zero-trust scenarios. We did implement a recycle-operation in case you’re the last user of an object. However, that was dropped due to horrible races in the kernel. It might reoccur in the future, though.
  • Graphics-Servers: This is a very specific use-case of IPC and usually there is a one-way trust relationship from clients to servers. However, a server cannot blindly trust its clients. So imagine a client renders its window-contents into memory and passes a file-descriptor to that memory region (maybe using memfd_create) to the server. Similar to the previous scenario, the server cannot mmap(2) that object for read-access as the client might truncate the file simultaneously, causing SIGBUS on the server. A server can protect itself via SIGBUS-handlers, but sealing is a much simpler way. By requiring F_SEAL_SHRINK, the server can be sure, the file will never shrink. At the same time, the client can still grow the object in case it needs bigger buffers for growing windows. Furthermore, writing is still allowed so the object can be re-used for the next frame.

As you might imagine, there are a lot more interesting use-cases. However, note that sealing is currently limited to objects created via memfd_create with the MFD_ALLOW_SEALING flag. This is a precaution to make sure we don’t break existing setups. However, changing seals of a file requires WRITE-access, thus it is rather unlikely that sealing would allow attacks that are not already possible with mandatory POSIX locks or similar. Hence, it is possible that sealing will expand to other areas in case people request it. Further seal-types are also possible.

Current Status

As of June 2014 the patches for memfd_create and sealing have been publicly available for at least 2 months and are considered for merging. linux-3.16 will probably not include it, but linux-3.17 very likely will. Currently, there’s still some issues to be figured out regarding AIO and Direct-IO races. But other than that, we’re good to go.

On Wifi-Display, Democratic Republics and Miracles

For a long time connecting your TV or other external monitors to your laptop has always been a hassle with cables and drivers. Your TV had an HDMI-port, but your laptop only DVI, your projector requires VGA and your MacBook can only do DP. Once you got a matching adapter, you noticed that the cable is far to short for you to sit comfortably on your couch. Fortunately, we’re about to put an end to this. At least, that was my impression when I first heard of Miracast.

Miracast is a certification program of the Wifi-Alliance based on their Wifi-Display specification. It defines a protocol to connect external monitors via wifi to your device. A rough description would be “HDMI over Wifi” and in fact the setup is modeled closely to the HDMI standard. Unlike existing video/audio-streaming protocols, Miracast was explicitly designed for this use-case and serves it very well. When I first heard of it about one year ago, I was quite amazed that it took us until late 2012 to come up with such a simple idea. And it didn’t take me long until I started looking into it.

For 4 months now I have been hacking on wifi, gstreamer and the kernel to get a working prototype. Despite many setbacks, I didn’t loose my fascination for this technology. I did complain a lot about the hundreds of kernel dead-locks, panics and crashes during development and the never-ending list of broken wifi-devices. However, to be fair I got the same amount of crashes and bugs on Android and competing proprietary Miracast products. I found a lot of other projects with the same goal, but the majority was closed-source, driver-specific, only a proof-of-concept or limited to sink-side. Nevertheless, I continued pushing code to the OpenWFD repository and registered as speaker for FOSDEM-2014 to present my results. Unfortunately, it took me until 1 week before FOSDEM to get the wifi-P2P link working realiably with an upstream driver. So during these 7 days I hacked up the protocol and streaming part and luckily got a working demo just in time. I never dared pushing that code to the public repository, though (among others it contains my birthday, which is surprisingly page-aligned, as magic mmap offset) and I doubt it will work on any setup but mine..

The first day after FOSDEM I trashed OpenWFD and decided to start all over again writing a properly integrated solution based on my past experience. Remembering the advice of a friend (“Projects with ‘Open’ in their name are just like countries named ‘Democratic Republic of …’”) I also looked for a new name. Given the fact that it requires a miracle to get Wifi-P2P working, it didn’t take me long to come up with something I liked. I hereby proudly announce: MiracleCast

MiracleCast

The core of MiracleCast is a daemon called miracled. It runs as system-daemon and manages local links, performs peer-discovery on request and handles protocol encoding and parsing. The daemon is independent of desktop environments or data transports and serves as local authority for all display-streaming protocols. While Miracast will stay the main target, other competing technologies like Chromecast, Airtame and AirPlay can be integrated quite easily.

The miraclectl command-line tool can be used to control the daemon. It can create new connections, modify parameters or destroy them again. It also supports an interactive mode where it displays events from miracled when they arrive, displays incoming connection attempts and allows direct user-input to accept or reject any requests.

miracled and miraclectl communicate via DBus so you can replace the command-line interface with graphical helpers. The main objects on this API are Links and Peers. Links describe local interfaces that are used to communicate with remote devices. This includes wifi-devices, but also virtual links like your local IP-Network. Peers are remote devices that were discovered on a local link. miracled hides the transport-type of each peer so you can use streaming protocols on-top of any available link-type (given the remote side supports the same). Therefore, we’re not limited to Wifi-P2P, but can use Ethernet, Bluetooth, AP-based Wifi and basically any other transport with an IP layer on top. This is especially handy for testing and development.

Local processes can now register Sources or Sinks with miracled. These describe capabilities that are advertised on the available links. Remote devices can then initiate a connection to your sink or, if you’re a source, you can connect to a remote sink. miracled implements the transmission protocol and hides it behind a DBus interface. It routes traffic from remote devices to the correct local process and vice versa. Regardless of whether the connection uses Miracast, Chromecast or plain RTSP, the same API is used.

Current Status

The main target still is Miracast! While all the APIs are independent of the protocol and transport layer, I want to get Miracast working as it is a nice way to test interoperability with Android. And for Miracast, we first need Wifi-P2P working. Therefore, that’s what I started with. The current miracled daemon implements a fully working Wifi-P2P user-space based on wpa_supplicant. Everyone interested is welcome to give it a try!

Everything on top, including the actual video-streaming is highly experimental and still hacked on. But if you hacked on Miracast, you know that the link-layer is the hard part. So after 1 week of vacation (that is, 1 week hacking on systemd!) I will return to MiracleCast and tackle local sinks.

As I have a lot more information than I could possible summarize here, I will try to keep some dummy-documentation on the wiki until I the first project release. Please feel free to contact me if you have any questions. Check out the git-repository if you want to see how it works! Note that MiracleCast is focused on proper desktop integration instead of fast prototyping, so please bear with me if API design takes some time. I’d really appreciate help on making Wifi-P2P work with as many devices as possible before we start spending all our efforts on the upper streaming layers.

During the next weeks, I will post some articles that explain step by step how to get Wifi-P2P working, how you can get some Miracast protoypes working and what competing technologies and implementations are available out there. I hope this way I can spread my fascination for Miracast and provide it to as many people as possible!

Splitting DRM and KMS device nodes

While most devices of the 3 major x86 desktop GPU-providers have GPU and display-controllers merged on a single card, recent development (especially on ARM) shows that rendering (via GPU) and mode-setting (via display-controller) are not necessarily bound to the same device. To better support such devices, several changes are being worked on for DRM.

In it’s current form, the DRM subsystem provides one general-purpose device-node for each registered DRM device: /dev/dri/card<num>. An additional control-node is also created, but it remains unused as of this writing. While in general a kernel driver is allowed to register multiple DRM devices for a single physical device, no driver made use of this, yet. That means, whatever hardware you use, both mode-setting and rendering is done via the same device node. This entails some rather serious consequences:

  1. Access-management to mode-setting and rendering is done via the same file-system node
  2. Mode-setting resources of a single card cannot be split among multiple graphics-servers
  3. Sharing display-controllers between cards is rather complicated

In the following sections, I want to look closer at each of these points and describe what has been done and what is still planned to overcome these restrictions. This is a highly technical description of the changes and serves as outline for the Linux-Plumbers session on this topic. I expect the reader to be familiar with DRM internals.

1) Render-nodes

While render-nodes have been discussed since 2009 on dri-devel, several mmap-related security-issues have prevented it from being merged. Those have all been fixed and 3-days ago, the basic render-node infrastructure has been merged. While it’s still marked as experimental and hidden behind the drm.rnodes module parameter, I’m confident we will enable it by default in one of the next kernel releases.

What are render-nodes?

From a user-space perspective, render-nodes are “like a big FPU” (krh) that can be used by applications to speed up computations and rendering. They are accessible via /dev/dri/renderD<num> and provide the basic DRM rendering interface. Compared to the old card<num> nodes, they lack some features:

  • No mode-setting (KMS) ioctls allowed
  • No insecure gem-flink allowed (use dma-buf instead!)
  • No DRM-auth required/supported
  • No legacy pre-KMS DRM-API supported

So whenever an application wants hardware-accelerated rendering, GPGPU access or offscreen-rendering, it no longer needs to ask a graphics-server (via DRI or wl_drm) but can instead open any available render node and start using it. Access-control to render-nodes is done via standard file-system modes. It’s no longer shared with mode-setting resources and thus can be provided for less-privileged applications.

It is important to note that render-nodes do not provide any new APIs. Instead, they just split a subset of the already available DRM-API off to a new device-node. The legacy node is not changed but kept for backwards-compatibility (and, obviously, for mode-setting).

It’s also important to know that render-nodes are not bound to a specific card. While internally it’s created by the same driver as the legacy node, user-space should never assume any connection between a render-node and a legacy/mode-setting node. Instead, if user-space requires hardware-acceleration, it should open any node and use it. For communication back to the graphics-server, dma-buf shall be used. Really! Questions like “how do I find the render-node for a given card?” don’t make any sense. Yes, driver-specific user-space can figure out whether and which render-node was created by which driver, but driver-unspecific user-space should never do that! Depending on your use-cases, either open any render-node you want (maybe allow an environment-variable to select it) or let the graphics-server do that for you and pass the FD via your graphics-API (X11, wayland, …).

So with render-nodes, kernel drivers can now provide an interface only for off-screen rendering and GPGPU work. Devices without any display-controller can avoid any mode-setting nodes and just provide a render-node. User-space, on the other hand, can finally use GPUs without requiring any privileged graphics-server running. They’re independent of the kernel-internal DRM-Master concept!

2) Mode-setting nodes

While splitting off render-nodes from the legacy node simplifies the situation for most applications, we didn’t simplify it for mode-setting applications. Currently, if a graphics-server wants to program a display-controller, it needs to be DRM-Master for the given card. It can acquire it via drmSetMaster() and drop it via drmDropMaster(). But only one application can be DRM-Master at a time. Moreover, only applications with CAP_SYS_ADMIN privileges can acquire DRM-Master. This prevents some quite fancy features:

  • Running an XServer without root-privileges
  • Using two different XServers to control two independent monitors/connectors of the same card

The initial idea (and Ilija Hadzic’s follow-up) to support this were mode-setting nodes. A privileged ioctl on the control-node would allow applications to split mode-setting resources across different device-nodes. You could have /dev/dri/modesetD1 and /dev/dri/modesetD2 to split your KMS CRTC and Connector resources. An XServer could use one of these nodes to program the now reduced set of resources. We would have one DRM-Master per node and we’d be fine. We could remove the CAP_SYS_ADMIN restriction and instead rely on file-system access-modes to control access to KMS resources.

Another discussed idea to avoid creating a bunch of file-system nodes, is to allocate these resources on-the-fly. All mode-setting-resources would now be bound to a DRM-Master object. An application can only access the resources available on the DRM-Master that it is assigned to. Initially, all resources are bound to the default DRM-Master as usual, which everyone gets assigned to when opening a legacy node. A new ioctl DRM_CLONE_MASTER is used to create a new DRM-Master with the same resources as the previous DRM-Master of an application. Via a DRM_DROP_MASTER_RESOURCE an application can drop KMS resources from their DRM-Master object. Due to their design, neither requires a CAP_SYS_ADMIN restriction as they only clone or drop privileges, they never acquire new privs! So they can be used by any application with access to the control node to create two new DRM-Master resources and pass them to two independent XServers. These use the passed FD to access the card, instead of opening the legacy or mode-setting nodes.

From the kernel side, the only thing that changes is that we can have multiple active DRM-Master objects. In fact, per DRM-Master one open-file might be allowed KMS access. However, this doesn’t require any driver-modifications (which were mostly “master-agnostic”, anyway) and only a few core DRM changes (except for vmwgfx-ttm-lock..).

3) DRM infrastructure

The previous two chapters focused on user-space APIs, but we also want the kernel-internal infrastructure to account for split hardware. However, fact is we already have anything we need. If some hardware exists without display-controller, you simply omit the DRIVER_MODESET flag and only set DRIVER_RENDER. DRM core will only create a render-node for this device then. If your hardware only provides a display-controller, but no real rendering hardware, you simply set DRIVER_MODESET but omit DRIVER_RENDER (which is what SimpleDRM is doing).

Yes, you currently get a bunch of unused DRM code compiled-in if you don’t use some features. However, this is not because DRM requires it, but only because no-one sent any patches for it, yet! DRM-core is driven by DRM-driver developers!

There is a reason why mid-layers are frowned upon in DRM land. There is no group of core DRM developers, but rather a bunch of driver-authors who write fancy driver-extensions. And once multiple drivers use them, they factor it out and move it to DRM core. So don’t complain about missing DRM features, but rather extend your drivers. If it’s a nice feature, you can count on it being incorporated into DRM-core at some point. It might be you doing most of the work, though!

Sane Session-Switching

In a previous article I talked about the history of VT switching. Created as a simple way to switch between text-mode sessions it has grown into a fragile API to protect one testosterone monster (also called XServer) from another. XServers used to poke in PCI bars, modified MMIO registers and messed around with DMA controllers. If they hauled out the big guns, it was almost absurd to believe a simple signal-flinging VT could ever successfully negotiate. Fortunately, today’s XServer is a repentant sinner. With common desktop hardware, all direct I/O is done in the kernel (thanks KMS!) and chances of screwing up your GPUs are rather minimal. This allows us to finally implement proper device-handover during session-switches.

If we look at sessions at a whole, the XServer isn’t special at all. A lot of session-daemons may run today that provide some service to the session as a whole. This includes pulseaudio, dbus, systemd –user, colord, polkit, ssh-keychain, and a lot more. All these daemons don’t need any special synchronization during session-switch. So why does the XServer require it?

Any graphics-server like the XServer is responsible of providing access to input and graphics devices to a session. When a session is activated, they need to re-initialize the devices. Before a session is deactivated, they need to cleanup the devices so the to-be-activated session can access them. The reason they need to do this is missing infrastructure to revoke their access. If a session would not cleanup graphics devices, the kernel would prevent any new session from accessing the graphics device. For input devices it is even worse: If a session doesn’t close the devices during deactivation, it would continue reading input events while the new session is active. So while typing in your password, the background session might send these key-strokes to your IRC client (which is exactly what XMir did). What we need is a kernel feature to forcibly revoke access to a graphics or input device. Unfortunately, it is not as easy at it sounds. We need to find some-one who is privileged and trusted enough to do this. You don’t want your background session to revoke your foreground session’s graphics access, do you? This is were systemd-logind enters the stage.

systemd-logind is already managing sessions on a system. It keeps track on which session is active and sets ACLs in /dev to give the foreground session access to device nodes. To implement device-handover, we extend the existing logind-API by a new function: RequestDevice(deviceNode). A graphics-server can pass a file-system path for a device-node in /dev to systemd-logind, which checks permissions, opens the node and returns a file-descriptor to the caller. But systemd-logind retains a copy of the file-descriptor. This allows logind to disable it as long as the session is inactive. During a session-switch, logind can now disable all devices of the old session, re-enable the devices of the new session and notify both of the session-switch. We now have a clean handover from one session to the other. With this technology in place, we can start looking at real scenarios.

1) Session-management with VTs

Session management using VTs for foreground control and logind for device management

Session management with VTs and logind

Based on the graphs for VT-switching, I drew a new one considering logind. VTs are still used to switch between sessions, but sessions no longer open hardware devices directly. Instead, they ask logind as described above. The big advantage is that VT-switches are no longer fragile. If a VT is active, it can be sure that it has exclusive hardware-access. And if a session is dead-locked, we can force a VT-switch and revoke their device-access. This allows to recover from situations where your XServer hangs without SSH’ing from a remote machine or using SysRq.

2) Session-management without VTs

Session management based solely on logind

Pure logind session management

While sane VT-switching is a nice feature, the biggest win is that we can implement proper multi-session support for seats without VTs. While previously only a single session could run on such seats, with logind device-management, we can now support session-switching on any seat.

Instead of using VTs to notify sessions when they are activated or deactivated, we use the logind-dbus-API. A graphics-server can now request input and graphics devices via the logind RequestDevice API and use it while active. Once a session-switch occurs, logind will disable the device file-descriptors and switch sessions. A dbus signal is sent asynchronously to the old and new session. The old session can stop rendering while inactive to save power.

3) Asynchonous events and backwards-compatibility

One thing changes almost unnoticed when using RequestDevice. An active graphics-server might be almost about to display an image on screen while a session-switch occurs. logind revokes access to graphics devices and sends an asynchronous event that the session is now inactive. However, the graphics-server might not have received this event, yet. Instead, it tries to invoke a system-call to update the screen. But this will fail with EACCES or EPERM as it doesn’t have access to it, anymore. Currently, for most graphics servers this is a fatal error. Instead of handling EACCES and interpreting it as “this device is now paused”, they don’t care for the error code and abort. We could fix all the graphics-servers, but to simplify the transition, we introduced negotiated session-switches.

Whenever logind is asked to perform a session-switch, it first sends PauseDevice signals for every open device to the foreground graphics-server. This must respond with a PauseDeviceComplete call to logind for each device. Once all devices are paused, the session-switch is performed. If the foreground session does not respond in a timely manner, logind will forcibly revoke device access and then perform the session-switch, anyway.

Note that negotiated session-switches are only meant for compatibility. Any graphics-server is highly encouraged to handle EACCES just fine!

All my local tests ran fine so far, but all this is still under development. systemd patches can be found at github (frequently rebased!). Most tests I do rely on an experimental novt library, also available at github (I will push it during next week; this is only for testing!). Feedback is welcome! The RFC can be found on systemd-devel. Now I need a day off..

Happy Switching!

How VT-switching works

Having multiple sessions on your system in parallel is a quite handy feature. It allows things like Fast User Switching or running two different DEs at the same time. Especially graphics developers like it, because they can test-run their experimental XServer/weston on the same machine they develop on.

To understand how it works, we need the concept of a session. See my previous introduction into session-management if you’re not familiar with it. I expect the reader to be familiar with basic session-management concepts (i.e., seats, systemd-logind, login-sessions, user-sessions, processes and daemons in a session).

1) Traditional text-mode VT-switching

Virtual terminals were introduced with linux-0.12. It’s the origin of multi-session support on linux. Before this, linux only supported a single TTY session (which was even available in the first tarball of linux-0.01). With virtual terminals, we have /dev/tty<num> devices, where <num> is between 1 and 63. They are always bound to seat0 and every session on seat0 is bound to a single VT. That means, there are at most 63 live sessions on seat0. Only one of them is active at a time (which can be read from /sys/class/tty/tty0/active).

Session management and device access with VTs in text-mode

Session management in text-mode

The kernel listens for keyboard events and switches between VTs on ctrl+alt+Fx shortcuts. Alternatively, you can issue a VT_ACTIVATE ioctl to politely ask the kernel to switch sessions.

In the old days, all sessions ran in text-mode. In text-mode, a process can get keyboard input by reading from a VT and can write onto the screen by writing to a VT. Some rather ugly control-sequences are supported to allow colors or other advanced features. The kernel interprets these and instructs the graphics hardware to print the given text.

Important to note is that in text-mode sessions don’t have direct hardware access. The kernel merges all keyboard events into one stream and a session cannot tell which device it came from. In fact, it cannot even tell how many devices there are. Same for graphics devices. As a VT has only a single output stream, there is only a single screen to write to. Multi-head support is not available. Neither are any advanced graphics-operations. But this allows the kernel to serialize access to hardware devices. Only the active VT gets input events and only the buffer of the active VT is displayed on the screen. No resource-conflicts can occur.

While there are ways to detect when a session is activated/deactivated, in text-mode a session normally doesn’t care. It just stops receiving keyboard input. If the session writes to the VT while deactivated, it will affect the internal buffer of the VT, but not the screen. Only the buffer of the active VT is shown on screen.

2) Session-switching in graphics-mode

System session management and device access with VTs in graphics mode

Session management in gfx-mode

Very soon it became clear that text-mode is not enough. We wanted more! That’s when the VT graphics mode was introduced. Graphics mode doesn’t change the setup, we still have 63 VTs and each session is bound to a VT. But a VT can now be switched from text-mode KD_TEXT into graphics-mode KD_GRAPHICS (via KDSETMODE ioctl). This doesn’t do anything spectacular. Really! The only effect is it disables the kernel-internal graphics routines. As long as a VT is in graphics-mode, the kernel will not instruct the graphics hardware to display the VT on screen. Once it is reset to text-mode and the VT is active, the kernel will display it again.

So the graphics-mode itself is useless. But at the same time, the kernel started providing separate interfaces to input and graphics devices. So while a VT is in graphics-mode, a session can access the graphics devices directly and render whatever they want. It can also ignore input from the VT and instead read input-events directly from the input-event interfaces. This is how the XServer works today.

But it is pretty obvious that this becomes problematic during session switches. If the kernel switches away from a graphics VT, the session needs to release the graphics hardware before the new session can be activated. Otherwise, the new session is active, but you still get the images from the old one. Or worse, it might flicker between the images of both sessions. Without kernel-mode-setting it might even hang your graphics hardware.

(A similar interface to KDSETMODE exists for keyboards with KDSKBMODE.)

Unfortunately, the kernel has no interface to forcibly revoke graphics or input access. So VTs were extended by the VT_SETMODE ioctl. A process can issue VT_SETMODE on a VT and pass two signal-numbers (usually SIGUSR1 and SIGUSR2). If the kernel wants to perform a VT-switch, it sends one such signal to the active VT. This VT can cleanup resources, stop using graphics/input devices and acknowledge the VT-switch via the VT_RELDISP ioctl. If the process dead-locked and doesn’t acknowledge the request, the VT-switch will not happen! This is why a crashed XServer can hang your system. But if it correctly issues the VT_RELDISP ioctl, the kernel will perform the VT-switch. Once the kernel switches back to the given VT it sends the second signal as notification that it is now active again.

However, on every VT (precisely, on every session) only a single process can call VT_SETMODE. This already shows that the concept is flawed. For example, if the XServer takes the VT in posession for graphics and input devices, another audio-server in the session couldn’t do the same for audio-hardware. This applies to all other devices. An alternative involving logind is discussed in a followup article.

3) Seats without VTs

We discussed that VTs are always bound to seat0. So if you run a session on a seat other than seat0 or if you disabled VTs entirely, then the situation becomes pretty simple: no multi-session support is available. This is enforced by systemd-logind so the active session can run without interruptions.

Also important to note is that there is no text-mode. A session must run in graphics mode as the kernel facility to interpret text-commands is bound to VTs.

Single-session setup on seats without VTs

Session management without VTs

4) logind integration

As we discussed in the previous article, today systemd-logind tracks and manages sessions. But how does this integrate with VTs? VTs pre-date systemd by years, so to preserve backwards-compatibility, we need to keep the infrastructure as it is. That’s why logind watches /sys/class/tty/tty0/active for changes. Once a VT switch happens, logind notices it and marks the old session as inactive and the new as active. It also adjusts ACLs in /dev to keep access-restrictions in sync with the active session. However, this cannot be done properly without a race-condition. Therefore, logind provides a new dbus-API to replace the ageing VT API. New session-daemons are advised to use it in favor of VTs.

Thoughts on Linux System Compositors

My work towards deprecating CONFIG_VT recently got stuck due to several kernel changes that are needed to proceed. DRM Render-nodes and Modeset-nodes are on their way and so I thought I’d provide a short update on what is being planned regarding session management in user-space.

This is no technical discussion or comparison between VTs and other session managers. Instead, I try to describe how we intend to proceed and why we need a new kind of user-space session manager. For technical details, feel free to browse through other articles in this blog or join the discussions on systemd-devel, wayland-devel and dri-devel MLs.

Virtual Terminals

While legacy systems with CONFIG_VT will always be supported, the current trend is towards a system without VTs. VTs are controversially discussed, but the major flaw of VTs is that they’re only available on the default seat. So if you’re a big fan of VTs, consider this whole article being about seats other than seat0, that is, no VTs are available! If you’re not so a big fan of VTs, join my dream of a world without VTs (and without fbcon, hurray!).

Systemd

With Ubuntu (and upstart) considering to emulate systemd-logind, it is becoming clear that the systemd-logind API is getting the de-facto standard for seat and session management on desktop linux systems. I don’t care what people think of systemd; for the purpose of this article I will call the session manager logind, feel free to replace it by your favorite seat/session manager.

Non-desktop systems are beyond the scope of this document. However, as long as you have a monitor+keyboard connected to your system, your setup is targeted by this article.

#1: The Problem

Most of the time, on a common linux desktop system, we interact with our favorite Desktop Environment; lets choose Gnome for today. Gnome manages the different screens, applications and tasks that we want to run. We can switch between them, stop them or start them. This is all managed inside a single Gnome-User-Session. The need for multiple sessions (and thus, multiple VTs) is barely noticeable to us. And that fact is very important! Whatever we do to extend session management, the single user session should still be the central part of the design. We must not make it, in any way slower, less stable, or harder to implement. Instead, we design our system around it.

So lets broaden our perspective and look at all the systems running in parallel to the user session: There is the boot-splash, linux-console, emergency-console, sleeping user-sessions, lock-screens, ssh-sessions, custom parallel Wayland/X11 sessions and probably a lot of crazy user-setups I am not even aware of. It is hard to work with such a diverse and moving target, however, they always have something in common: Only one desktop session is active at a time!

But which one?

#2: logind to the rescue

Layout and drawbacks of the VT Session Manager

It is pretty obvious that we need an IPC mechanism to notify the different sessions about who is active and when to switch between them. Historically, VTs provided a decentralized way by letting each session open a different VT and making the active VT responsible of session-management. To be fair, there is support for automatic session-switching, but most sessions take control over input devices and thus need to schedule session switches themselves. Major flaw is that if the active session deadlocks, your system becomes unresponsive. While the VT system has several other drawbacks, discussion is beyond the scope of this document as we always have the situation where we don’t have VTs (think: seats other than seat0).

Advanced access control with FD passing via systemd-logind

Without VTs, we are currently left with only a single session per seat. Pretty boring! And that’s probably also what the desktop group at RedHat thought. A broadly discussed idea from several RedHat developers is to extend logind to arbitrate between the different sessions (Note: this article is about my perception, but the idea of using logind was originally not from me). logind already knows about all sessions on a system, knows about all seats and is the perfect place to implement session management. So instead of opening a VT, sessions now use dbus to tell logind that they provide a new user session. The API hasn’t been figured out, yet, but the big advantage over VTs is that we can pass file-descriptors directly to each session. logind can mute input-devices (patches pending on linux-input ML) for inactive sessions and can revoke DRM access. It guarantees that the active session has exclusive hardware access even if a session dead-locks.

Left for discussion is the design of the logind dbus API. It will include a set of calls to register, activate, deactivate and unregister sessions. Due to the fact that we have control over hardware resources, we can also implement forced session switching. Moreover, we could make logind listen to magic sysrq keys which deactivates the current session and instead activates the emergency console.

#3: Limitations

logind is no system compositor. logind will never be a system compositor. In fact, if logind becomes the central session manager, the concept of a system compositor will be dead and buried. The problem is, a session manager requires exclusive access over any sessions and hardware. Same is true for a system-compositor. Both concepts don’t play nice together. But what are the benefits of a system-compositor over logind?

While anything a system-compositor can do, could theoretically be implemented in logind, we assume that logind is supposed to stay small and efficient. So logind will never provide any DRM display handling. All sessions must implement it themselves. Same holds true for input handling. logind can only provide FD forwarding, but the heavy lifting must be done directly in each session program. In every small session program! The boot-splash must include complex DRM and XKB code, the screen-lock, too. Even the emergency console has to do all the low-level hardware handling. This introduces complexity and, very likely, bugs. But on the other hand, it guarantees efficiency. An xserver can run with direct hardware access and no extra context-switch to logind is needed on every frame.

But DRM handling could be simplified via a shared library for every session that doesn’t need custom 3D acceleration. So what else do we abandon for the sake of simplicity?

Probably the most hurtful is that global keyboard management is impossible. We cannot make logind listen for keyboard events (other than real emergency shortcuts) at the same time as a session is active. Otherwise, we might end up with logind and the session reacting to a single shortcut, which is unacceptable in any non-emergency situation. So session-switching will be left to the active-session itself. But if the active session dead-locks, we have the same problem as with VTs: an unresponsive system

We can implement emergency-shortcuts which violate this rule, but it will never be a clean solution. Moreover, we will never be able to have global Volume-Up and Volume-Down shortcut handling, no Brightness-keys, no global NumLock handling. Global keyboard handling will be reliant on policies, if implemented at all, and otherwise left to each session to implement.

#4: Quirks

Layout of different quirks working with logind session manager

Upgrading logind to a session-manager has advantages on its own. Most of the code is already available, no external manager is needed, which would have to synchronize with logind. We have direct hardware access in each session and, as mentioned earlier, sessions are self-hosting. A session itself doesn’t need any major modifications to run under this scenario. A session controls itself over the whole runtime and, more importantly, doesn’t depend on another daemon to function properly. Yes, session-management depends on logind, but the session itself, once activated, runs standalone.

To overcome the issues mentioned above, I developed a set of quirks that sit between logind and a session:

UVTD

uvtd (User-Space Virtual-Terminal Daemon) is an implementation of VTs in user-space. It is available as experimental branch in the official kmscon repository. Even if you disable kernel VTs, uvtd can provide legacy VT nodes for backwards compatibility. But in contrast to kernel VTs, uvtd can control the “server side” of each VT and provide a bridge between a fake VT and logind. This allows to run legacy VT applications even if no VTs are available. The current code already allows to run multiple xservers in parallel without any real VT and without modifying a single xserver line of code.

wlsystemc

wlsystemc (Wayland System Compositor) is another research project I was working on. It is a Wayland compositor that displays each client as fullscreen surface. Due to the fast moving Wayland codebase it got slightly out of date during the last year, but it was once working with basic SHM buffers. Anyhow, together with logind, wlsystemc can be redesigned to take over modesetting for basic sessions. For instance, a boot-splash would create a surface in wlsystemc, which itself registers a session in logind for this surface. Once the session gets activated, wlsystemc performs all the heavy DRM operations and the boot-splash does nothing more than simple generic Wayland rendering (i.e., passing custom buffer).

Kernel Log

“No VTs, no kernel log!”. But especially developers want a kernel oops or panic printed on screen whenever they occur. The solution is fblog or drmlog, a kernel module, that does nothing else than printing the kernel log to all connected monitors. logind can register it like any other session and enable or disable it via sysfs. During oops/panic the kernel will activate them automatically and notify logind asynchronously via sysfs. Both modules are still under development, but at least fblog is already working.

Linux Console

For completeness: kmscon can easily replace the linux-console in this setup. It will register a text-console as a normal session just like any other session does.

#5: Conclusion

While I was an enthusiastic advocate of the system-compositor idea, it did become pretty clear, that the performance penalty for input-event forwarding and the huge required kernel API extensions make it very unlikely that it will get accepted in the community. Furthermore, besides global keyboard handling, most technical concerns on the logind idea can be solved with quirks like wlsystemc and uvtd.

This post didn’t discuss or highlight the low-level technical concerns on kernel VTs, but I hope it explains why we get a lot more flexibility and possibilities by moving session handling into user-space. logind can be taught boot-splash awareness, emergency-handling or forced session-switching, which will be very unlikely to get implemented in kernel VTs.

But independent of how we implement it, moving VT handling to userspace and disabling CONFIG_VT still has one major advantage: it simplifies kernel programming a lot. And I know, nearly every DRM developer would be very happy to see this happen!

DRM Render- and Modeset-Nodes

Another year, another Google Summer of Code. This time I got the chance to work on something that I had on my TODO list for quite a long time: DRM Render- and Modeset-Nodes

As part of the X.org Foundation mentoring organization, I will try to pick up the work from Ilija HadzicDave AirlieKristian HoegsbergMartin Peres and others. The idea is to extend the DRM user-space API of the linux kernel to split modeset and rendering interfaces apart. The main usage is to allow different access-modes for graphics-compositors (which require the modeset API) and client-side rendering or GPGPU-users (which both require the rendering API). We currently use the DRM-Master interface to restrict the modeset API to privileged applications. However, this requires SYS_CAP_ADMIN privileges, which is roughly equivalent to root-privileges. With two different interfaces for modeset and rendering APIs, we can apply a different set of filesystem-access-modes to each of them and thus get fine-grained access-control.

Apart from fine-grained access control, we also get some other nice features almost for free:

  • We will be able to run GPGPU clients without any running compositor or event without any display controller
  • We can split modeset objects across multiple nodes to allow multi-seat setups with a single display controller
  • Efficient compositor-stacking by granting page-flip access or full modeset access temporarily to sub-compositors

There are actually a lot of other ideas how to extend this. So I decided to concentrate on the modeset-node / render-node split first. Once that is done (and fully working), I will pick different ideas that depend on this and try to implement them. Considering the lot of work others have put in this already, I think if I get the split merged into mainline, the project will already be a great success. Everything on top of it will be some bonus that will probably take more time to get merged. But lets see, maybe it turns out to be easier than I think and we end up with some of the use-cases merged upstream, too.

Thanks to the X.org Foundation, Google and my GSoC-mentor Dave Airlie for giving me the chance to work on the DRM API! I hope it will be a productive summer.

GSoC-Proposal

If someone is interested in more details, some excerpts from my original GSoC proposal:

Project Description:
Since several years xserver is no longer the only user-space project that makes
use of the kernel DRM API. The introduction of KMS allowed many new projects to
emerge, including plymouth, weston and kmscon. On the other side, OpenCL support
allows applications to make use of DRM without requiring any KMS APIs. Even though
both use-cases work with the current APIs, there are a lot of restrictions that
need to be worked around.

The most problematic concept is DRM-Master. KMS applications are required to be
DRM-Master to perform modesetting, but DRM-Master is tightly coupled to
CAP_SYS_ADMIN/root. On the other side, render clients are required to be assigned
to a DRM-Master so they can get authenticated. This prevents off-screen/offline
rendering without a running compositor.

One possible solution is to split render- and modeset-nodes apart. The DRM control
node can be used as the management node (which is, as far as I understand, what
it was designed for, anyway). A separate static render-node is created for each
DRM device which is restricted to ioctls specific to rendering operations. Instead
of requiring drmAuth() for authorization, we can now use filesystem access-modes.
This allows slightly more dynamic access-control, but the biggest advantage is
that we can do off-screen/offline rendering without a running
compositor/DRM-Master.

On the other side, a modeset-node is a concept to have KMS separated from DRM.
The use-case is to split modeset objects (eg., crtcs, encoders, connectors) across
different modesetting applications. This allows one compositor to use one
CRTC+connector combination, while another compositor (maybe on another seat) can
use another CRTC+external-connector. This doesn't have to be a static setup. On
the contrary, one use-case I am very interested in is a dynamic modeset-object
assignment to temporary clients. This way, a fullscreen application can be granted
page-flip rights from the compositor to avoid context-switches to the compositor
for doing trivial page-flips only.

Deliverables
* Working render-node clients: Preferably an offline OpenCL example and
  a wayland EGL client
* Merged kernel render-node implementation with at least i915 support
* Dynamic kernel modeset-nodes
* "Zero-context-switches" wayland/weston fullscreen client (optional)

Known Problems:
There were several attempts to push render-nodes into the kernel, but all failed
due to missing motivation to finish the user-space clients. Writing up new fancy
APIs is one part, but pushing API changes to such big projects requires the whole
environment to work well with the changes. That's why I want to concentrate on
the user-space side of render-nodes. And I want to finish the render-nodes project
before continuing with modeset-nodes. The idea has been around long enough that
it's time that we get it done.

However, one problem is that I never worked with the low-level X11 stack. The
wayland environment is great for experiments and quite active. I am very familiar
with it and know how to get examples easily running. The xserver, however, is a
huge black box to me. I know the concepts and understand the input and graphics
drivers design. But I never read xserver core code. That's something I'd like to
change during this project. I will probably be limited to DRI and graphics
drivers, but that's a good start.

Another idea that came up quite often is something like gem-fs. It's far beyond
the scope of this project, but it's something I'd like to keep in mind when
designing the API. It's hard to account for something that's only an idea, but
the concepts seem related so I will try to understand the reasons behind gem-fs
and avoid orthogonal implementations.

A few smaller implementation-specific problems are already known, including the
mmap-security problem, static "possible_encoders"/"possible_crtcs" bitsets and
missing MMUs on GPUs. However, there already have been ideas how to solve them
so I don't consider them blockers for render-nodes.

Linux Wii Remote Driver Updates

Linux Bluetooth HID handling has been slightly broken in the kernel since several years. Reconnecting devices caused a kernel oops nearly every time you tried. Two months ago I sat down and rewrote the HIDP session handling and fixed several bugs. The series now got merged and will be part of linux-3.10. If you still encounter bugs please leave me a note.

Based on this I tried recovering my hid-wiimote work. Since the last time I reverse-engineered Nintendo devices, a lot has happened. The most significant change is probably the release of the Wii U. While the Gamepad is still an ongoing target for r/e, other devices were much easier to get working. Motivated by the new hardware I just got, I finally figured out a reliable way for extension hotplugging. This wasn’t supported in the kernel until now and required nasty polling-techniques to work around race-conditions in the proprietary protocol (who designs asynchronous protocols without protocol barriers?). So I rewrote most of the core device handling and moved the input parsers to a module based infrastructure. This allows us to easily extend the hid-wiimote driver to support new devices that are based on the same protocol.

The series is still pending on the linux-input ML but I hope to get basic hotplugging support into linux-3.10. Further support for the built-in speaker device or the Wii U Pro Controller will hopefully follow with 3.11.

The xwiimote user-space stack has already been updated and I will push the changes this weekend.