Sane Session-Switching

In a previous article I talked about the history of VT switching. Created as a simple way to switch between text-mode sessions it has grown into a fragile API to protect one testosterone monster (also called XServer) from another. XServers used to poke in PCI bars, modified MMIO registers and messed around with DMA controllers. If they hauled out the big guns, it was almost absurd to believe a simple signal-flinging VT could ever successfully negotiate. Fortunately, today’s XServer is a repentant sinner. With common desktop hardware, all direct I/O is done in the kernel (thanks KMS!) and chances of screwing up your GPUs are rather minimal. This allows us to finally implement proper device-handover during session-switches.

If we look at sessions at a whole, the XServer isn’t special at all. A lot of session-daemons may run today that provide some service to the session as a whole. This includes pulseaudio, dbus, systemd –user, colord, polkit, ssh-keychain, and a lot more. All these daemons don’t need any special synchronization during session-switch. So why does the XServer require it?

Any graphics-server like the XServer is responsible of providing access to input and graphics devices to a session. When a session is activated, they need to re-initialize the devices. Before a session is deactivated, they need to cleanup the devices so the to-be-activated session can access them. The reason they need to do this is missing infrastructure to revoke their access. If a session would not cleanup graphics devices, the kernel would prevent any new session from accessing the graphics device. For input devices it is even worse: If a session doesn’t close the devices during deactivation, it would continue reading input events while the new session is active. So while typing in your password, the background session might send these key-strokes to your IRC client (which is exactly what XMir did). What we need is a kernel feature to forcibly revoke access to a graphics or input device. Unfortunately, it is not as easy at it sounds. We need to find some-one who is privileged and trusted enough to do this. You don’t want your background session to revoke your foreground session’s graphics access, do you? This is were systemd-logind enters the stage.

systemd-logind is already managing sessions on a system. It keeps track on which session is active and sets ACLs in /dev to give the foreground session access to device nodes. To implement device-handover, we extend the existing logind-API by a new function: RequestDevice(deviceNode). A graphics-server can pass a file-system path for a device-node in /dev to systemd-logind, which checks permissions, opens the node and returns a file-descriptor to the caller. But systemd-logind retains a copy of the file-descriptor. This allows logind to disable it as long as the session is inactive. During a session-switch, logind can now disable all devices of the old session, re-enable the devices of the new session and notify both of the session-switch. We now have a clean handover from one session to the other. With this technology in place, we can start looking at real scenarios.

1) Session-management with VTs

Session management with VTs and logind

Based on the graphs for VT-switching, I drew a new one considering logind. VTs are still used to switch between sessions, but sessions no longer open hardware devices directly. Instead, they ask logind as described above. The big advantage is that VT-switches are no longer fragile. If a VT is active, it can be sure that it has exclusive hardware-access. And if a session is dead-locked, we can force a VT-switch and revoke their device-access. This allows to recover from situations where your XServer hangs without SSH’ing from a remote machine or using SysRq.

2) Session-management without VTs

Pure logind session management

While sane VT-switching is a nice feature, the biggest win is that we can implement proper multi-session support for seats without VTs. While previously only a single session could run on such seats, with logind device-management, we can now support session-switching on any seat.

Instead of using VTs to notify sessions when they are activated or deactivated, we use the logind-dbus-API. A graphics-server can now request input and graphics devices via the logind RequestDevice API and use it while active. Once a session-switch occurs, logind will disable the device file-descriptors and switch sessions. A dbus signal is sent asynchronously to the old and new session. The old session can stop rendering while inactive to save power.

3) Asynchonous events and backwards-compatibility

One thing changes almost unnoticed when using RequestDevice. An active graphics-server might be almost about to display an image on screen while a session-switch occurs. logind revokes access to graphics devices and sends an asynchronous event that the session is now inactive. However, the graphics-server might not have received this event, yet. Instead, it tries to invoke a system-call to update the screen. But this will fail with EACCES or EPERM as it doesn’t have access to it, anymore. Currently, for most graphics servers this is a fatal error. Instead of handling EACCES and interpreting it as “this device is now paused”, they don’t care for the error code and abort. We could fix all the graphics-servers, but to simplify the transition, we introduced negotiated session-switches.

Whenever logind is asked to perform a session-switch, it first sends PauseDevice signals for every open device to the foreground graphics-server. This must respond with a PauseDeviceComplete call to logind for each device. Once all devices are paused, the session-switch is performed. If the foreground session does not respond in a timely manner, logind will forcibly revoke device access and then perform the session-switch, anyway.

Note that negotiated session-switches are only meant for compatibility. Any graphics-server is highly encouraged to handle EACCES just fine!

All my local tests ran fine so far, but all this is still under development. systemd patches can be found at github (frequently rebased!). Most tests I do rely on an experimental novt library, also available at github (I will push it during next week; this is only for testing!). Feedback is welcome! The RFC can be found on systemd-devel. Now I need a day off..

Happy Switching!

Advertisement

Session-Management on Linux

One thing I always admired on linux is fast session-switching. A simple key-press (ctrl+alt+Fx) and I can ditch my current session to switch to another one. While in principle session-interaction is fairly simple, you can easily get it wrong (as can be seen with XMir). But how exactly does session management work today? And how did systemd-logind change this situation?

First, we need to define a session. A session is a group of processes running under control of a single user. Only a single session is active on a system (more precisely on a seat; discussed below) and a user can only interact with the active (or foreground) session. A session can start and stop processes, but it cannot escape from the session. In other words, all started processes will always belong to the session they were started in. Only system-daemons (which are not attached to a session) can spawn new sessions.

A session usually has different user-daemons which provide services for the session. For instance, the X-Server provides graphics access to other session-processes, pulseaudio provides audio-access. Then there is a central process which controls and monitors a session. Whenever a session is spawned, this process is the first process that is created. It initializes the session, spawns other management processes and optionally restarts them if they crash. It might also exit once the session was initialized if no process-monitoring is required. An example would be gnome-session.

1) Session Management

Historically, every process on a system has a session-id which defines the session a process belongs to. The setsid() system call allows to create a new session. However, with systemd-logind a new more-versatile concept was introduced. To understand this, we need to first look at seats.

Seats

Multi-seat setup (source: Wikipedia)

On normal systems only a single seat exists (called seat0). But more seats can be created on request. A seat is a virtual object that describe a physical interface to a system. The combination of a monitor plus keyboard and mouse form a usual seat. One user can interact with the system through this seat. If you attach another monitor plus keyboard and mouse, another user can interact with the system in parallel via this new seat (eg., called seat1). It is important to note that these seats are independent of each other. The pictures you get on the monitors are different and keyboard input from one seat doesn’t affect the other.

Any device connected to your system can be attached to a seat, but only to one seat at a time! If a device is attached to seat, it cannot be used by processes running on another seat. Obviously, input devices, sound-devices and monitors are usually attached to a seat. Network-interfaces, for example, can be left unattached so all seats can share an internet-connection. It is up to the system-administrator to decide which devices to attach to which seat, or whether to leave them as global devices that can be shared.

Bootup

During system-boot, systemd starts several daemons to manage a system. All these daemons are global and not attached to a seat or session. This also means, these daemons must not access devices attached to a seat. Among these daemons is also systemd-logind, which is responsible for session and seat management.

To allow user interaction, systemd needs to automatically spawn a new session for each seat. This is usually a login-session (like gdm, kdm or LightDM) but might also be configured as auto-login and spawn directly a user-session like gnome-session. In case of a login-session, the greeter draws login and password fields and after verifying the input, it instructs systemd to spawn a user-session. Note that the greeter cannot start the user-session itself as it already runs in a session (you cannot escape your session, remember?).

But what exactly does systemd do to control sessions?

Sessions

Via kernel interfaces (namely cgroups) systemd can reliably track processes and their childs. Whenever systemd is instructed to start a new session, it first verifies the caller is allowed to do that and then it spawns the first process for the session. Internally, it attaches a new session-id to this process, saves the seat this session is run on and some other maintenance data. Whenever this session process creates new processes, systemd can use cgroups to track these. So it always knows which session a process belongs to.

The first process for a session is responsible of initializing the session. In case of gnome-session, it spawns the X-Server, some gnome-specific maintenance daemons and then monitors the session in case something goes wrong. You can now interact with this session (normally through the X-Server) and do your daily work.

A session can explicitly instruct systemd to close itself. However, this will only mark it as dead. Once all processes of a session exited, the session is automatically cleaned up and removed.

2) Case-Study: Text-mode sessions

Text-mode interface (source: Wikipedia)

As a concrete non-trivial scenario, I’ll explain how text-mode sessions integrate with this. First, we need to know that virtual-terminals are basically a combination of input devices and a monitor. They can be accessed via /dev/tty<num> (where <num> is between 1 and 63; the other /dev/ttyXY devices are no VTs!). If you read from them, you get input from connected keyboards, if you write to them, the kernel displays it as text on the monitor. Some rather complex control-sequences are available to do colors or more, but that’s beyond the scope. VTs are always bound to seat0. So only sessions on this seat can use VTs (which means, classic text-mode is only available on seat0).

Text-mode sessions are special in that they are not spawned by a login-session. systemd-logind spawns /bin/agetty right during boot for each VT (you can configure how many of the 63 possible should be started). agetty is running as system daemon and not in a session! However, it is bound to seat0, obviously. agetty initializes the VT and runs /bin/login. login then writes a greeter to the VT and reads username plus password from it. As you might notice, this is problematic as it accesses devices attached to a seat even though it’s not running in a session. But due to the design of VTs, this cannot be easily avoided.

Once login verified the user-input, it instructs systemd to start the given user-session in text-mode. It looks up the initial process in /etc/passwd (let’s assume it is /bin/bash) and runs it. So bash is the controlling process in this session. It also substitutes the X-Server in that it provides user-interaction (in text mode rather than graphics). It allows to switch between different processes (ctrl+Z, fg, bg and jobs) and allows starting other session-daemons during initialization (via .bashrc).

So while text-mode feels a lot different, it is in fact very similar to graphics mode and is internally handled almost equally.

3) Multi-Session

We now understand what a session is, how they are created and how a user interacts with them. However, we haven’t discussed what happens if there are multiple sessions on a seat.

On current systems, multiple sessions are only allowed on seat0 if VTs are enabled (which is usually the case). That means, on all other seats, once you spawned a session, you have to stick with it. You cannot ctrl+alt+Fx away from the session (the reason for this is historical and we’re about to change this). However, you can close the session and then start a new one.

But if we assume our seat supports multiple sessions, how does that work? systemd-logind is responsible to manage and track sessions. Therefore, it keeps an Active attribute for each session. It’s a boolean value that defines whether the session is currently active or not. systemd-logind takes care that always at most one session is active. If no session runs on a seat, no session might be active (also in other situations which we ignore here as it’s an implementation detail). Additionally, systemd-logind sets the correct access-restrictions on devices attached to a seat so only the active session can access these devices. This guarantees that no background session interferes with foreground activity.

To switch between sessions, each session has to listen for special keyboard input (normally ctrl+alt+Fx) and instruct systemd-logind to switch to another session once it’s pressed. Today, this is implemented in the X-Server or, if no x-server runs in the session, by the underlying VT. You can also send a dbus call to systemd-logind directly to perform the session-switch (see loginctl activate <sid>).

So for proper multi-session support, a login-session spawns a user-session during login but stays active in background. A user can now switch between the user-session and the login-session and optionally spawn additional session by logging in again. Session processes can listen for dbus signals from systemd-logind to be notified when they are activated or deactivated. This allows them to go asleep while being in background and start up again once being put in foreground.

For an in-depth view on session-switching, see this followup article.