My work towards deprecating CONFIG_VT recently got stuck due to several kernel changes that are needed to proceed. DRM Render-nodes and Modeset-nodes are on their way and so I thought I’d provide a short update on what is being planned regarding session management in user-space.
This is no technical discussion or comparison between VTs and other session managers. Instead, I try to describe how we intend to proceed and why we need a new kind of user-space session manager. For technical details, feel free to browse through other articles in this blog or join the discussions on systemd-devel, wayland-devel and dri-devel MLs.
Virtual Terminals
While legacy systems with CONFIG_VT will always be supported, the current trend is towards a system without VTs. VTs are controversially discussed, but the major flaw of VTs is that they’re only available on the default seat. So if you’re a big fan of VTs, consider this whole article being about seats other than seat0, that is, no VTs are available! If you’re not so a big fan of VTs, join my dream of a world without VTs (and without fbcon, hurray!).
Systemd
With Ubuntu (and upstart) considering to emulate systemd-logind, it is becoming clear that the systemd-logind API is getting the de-facto standard for seat and session management on desktop linux systems. I don’t care what people think of systemd; for the purpose of this article I will call the session manager logind, feel free to replace it by your favorite seat/session manager.
Non-desktop systems are beyond the scope of this document. However, as long as you have a monitor+keyboard connected to your system, your setup is targeted by this article.
#1: The Problem
Most of the time, on a common linux desktop system, we interact with our favorite Desktop Environment; lets choose Gnome for today. Gnome manages the different screens, applications and tasks that we want to run. We can switch between them, stop them or start them. This is all managed inside a single Gnome-User-Session. The need for multiple sessions (and thus, multiple VTs) is barely noticeable to us. And that fact is very important! Whatever we do to extend session management, the single user session should still be the central part of the design. We must not make it, in any way slower, less stable, or harder to implement. Instead, we design our system around it.
So lets broaden our perspective and look at all the systems running in parallel to the user session: There is the boot-splash, linux-console, emergency-console, sleeping user-sessions, lock-screens, ssh-sessions, custom parallel Wayland/X11 sessions and probably a lot of crazy user-setups I am not even aware of. It is hard to work with such a diverse and moving target, however, they always have something in common: Only one desktop session is active at a time!
But which one?
#2: logind to the rescue
It is pretty obvious that we need an IPC mechanism to notify the different sessions about who is active and when to switch between them. Historically, VTs provided a decentralized way by letting each session open a different VT and making the active VT responsible of session-management. To be fair, there is support for automatic session-switching, but most sessions take control over input devices and thus need to schedule session switches themselves. Major flaw is that if the active session deadlocks, your system becomes unresponsive. While the VT system has several other drawbacks, discussion is beyond the scope of this document as we always have the situation where we don’t have VTs (think: seats other than seat0).
Without VTs, we are currently left with only a single session per seat. Pretty boring! And that’s probably also what the desktop group at RedHat thought. A broadly discussed idea from several RedHat developers is to extend logind to arbitrate between the different sessions (Note: this article is about my perception, but the idea of using logind was originally not from me). logind already knows about all sessions on a system, knows about all seats and is the perfect place to implement session management. So instead of opening a VT, sessions now use dbus to tell logind that they provide a new user session. The API hasn’t been figured out, yet, but the big advantage over VTs is that we can pass file-descriptors directly to each session. logind can mute input-devices (patches pending on linux-input ML) for inactive sessions and can revoke DRM access. It guarantees that the active session has exclusive hardware access even if a session dead-locks.
Left for discussion is the design of the logind dbus API. It will include a set of calls to register, activate, deactivate and unregister sessions. Due to the fact that we have control over hardware resources, we can also implement forced session switching. Moreover, we could make logind listen to magic sysrq keys which deactivates the current session and instead activates the emergency console.
#3: Limitations
logind is no system compositor. logind will never be a system compositor. In fact, if logind becomes the central session manager, the concept of a system compositor will be dead and buried. The problem is, a session manager requires exclusive access over any sessions and hardware. Same is true for a system-compositor. Both concepts don’t play nice together. But what are the benefits of a system-compositor over logind?
While anything a system-compositor can do, could theoretically be implemented in logind, we assume that logind is supposed to stay small and efficient. So logind will never provide any DRM display handling. All sessions must implement it themselves. Same holds true for input handling. logind can only provide FD forwarding, but the heavy lifting must be done directly in each session program. In every small session program! The boot-splash must include complex DRM and XKB code, the screen-lock, too. Even the emergency console has to do all the low-level hardware handling. This introduces complexity and, very likely, bugs. But on the other hand, it guarantees efficiency. An xserver can run with direct hardware access and no extra context-switch to logind is needed on every frame.
But DRM handling could be simplified via a shared library for every session that doesn’t need custom 3D acceleration. So what else do we abandon for the sake of simplicity?
Probably the most hurtful is that global keyboard management is impossible. We cannot make logind listen for keyboard events (other than real emergency shortcuts) at the same time as a session is active. Otherwise, we might end up with logind and the session reacting to a single shortcut, which is unacceptable in any non-emergency situation. So session-switching will be left to the active-session itself. But if the active session dead-locks, we have the same problem as with VTs: an unresponsive system
We can implement emergency-shortcuts which violate this rule, but it will never be a clean solution. Moreover, we will never be able to have global Volume-Up and Volume-Down shortcut handling, no Brightness-keys, no global NumLock handling. Global keyboard handling will be reliant on policies, if implemented at all, and otherwise left to each session to implement.
#4: Quirks
Upgrading logind to a session-manager has advantages on its own. Most of the code is already available, no external manager is needed, which would have to synchronize with logind. We have direct hardware access in each session and, as mentioned earlier, sessions are self-hosting. A session itself doesn’t need any major modifications to run under this scenario. A session controls itself over the whole runtime and, more importantly, doesn’t depend on another daemon to function properly. Yes, session-management depends on logind, but the session itself, once activated, runs standalone.
To overcome the issues mentioned above, I developed a set of quirks that sit between logind and a session:
UVTD
uvtd (User-Space Virtual-Terminal Daemon) is an implementation of VTs in user-space. It is available as experimental branch in the official kmscon repository. Even if you disable kernel VTs, uvtd can provide legacy VT nodes for backwards compatibility. But in contrast to kernel VTs, uvtd can control the “server side” of each VT and provide a bridge between a fake VT and logind. This allows to run legacy VT applications even if no VTs are available. The current code already allows to run multiple xservers in parallel without any real VT and without modifying a single xserver line of code.
wlsystemc
wlsystemc (Wayland System Compositor) is another research project I was working on. It is a Wayland compositor that displays each client as fullscreen surface. Due to the fast moving Wayland codebase it got slightly out of date during the last year, but it was once working with basic SHM buffers. Anyhow, together with logind, wlsystemc can be redesigned to take over modesetting for basic sessions. For instance, a boot-splash would create a surface in wlsystemc, which itself registers a session in logind for this surface. Once the session gets activated, wlsystemc performs all the heavy DRM operations and the boot-splash does nothing more than simple generic Wayland rendering (i.e., passing custom buffer).
Kernel Log
“No VTs, no kernel log!”. But especially developers want a kernel oops or panic printed on screen whenever they occur. The solution is fblog or drmlog, a kernel module, that does nothing else than printing the kernel log to all connected monitors. logind can register it like any other session and enable or disable it via sysfs. During oops/panic the kernel will activate them automatically and notify logind asynchronously via sysfs. Both modules are still under development, but at least fblog is already working.
Linux Console
For completeness: kmscon can easily replace the linux-console in this setup. It will register a text-console as a normal session just like any other session does.
#5: Conclusion
While I was an enthusiastic advocate of the system-compositor idea, it did become pretty clear, that the performance penalty for input-event forwarding and the huge required kernel API extensions make it very unlikely that it will get accepted in the community. Furthermore, besides global keyboard handling, most technical concerns on the logind idea can be solved with quirks like wlsystemc and uvtd.
This post didn’t discuss or highlight the low-level technical concerns on kernel VTs, but I hope it explains why we get a lot more flexibility and possibilities by moving session handling into user-space. logind can be taught boot-splash awareness, emergency-handling or forced session-switching, which will be very unlikely to get implemented in kernel VTs.
But independent of how we implement it, moving VT handling to userspace and disabling CONFIG_VT still has one major advantage: it simplifies kernel programming a lot. And I know, nearly every DRM developer would be very happy to see this happen!
Ponyhof 
Very nice article, those seem to be sensible ideas 🙂
fyi, the third picture (quirks) doesn’t have a link, the other two do.
Thanks! And I fixed the link.
One element that is useful for me is that the VT allowed to rescue whatever happened in main session. For me it was mostly ‘gnome-shell’ crashed due to old version of spidermonkey (before 1.8.8 was released). While it can be argued it is used by power users only it is also used (by power users) to rescue normal users systems (legal ‘backdoor’, in sense of accessing internals of computer, not security, can be quite useful).
Since I won’t pretend that I understood fully the article (yet) – would there be any Ctrl+Alt+F1 (or anything similar) to get to the ‘rescue’ console enabled by default (or to use when say wayland is crashing for whatever reason)?
I think kmscon lets you do that.
As Stephen already said, that is still possible. Having VT management in the kernel does allow reacting to keyboard events in interrupt handlers, however, you need at least a working bash to make use of an emergency console. And if bash can run, kmscon can run, too. In fact, the fbcon->fbdev->DRM layer is known to be fragile in those situations, so we could even end up with a more reliable emergency console.
If bash can run kmscon can run too? That is way too big an assumption. It all depends on what’s in development. Bash depends on glibc and readline. Kmscon depends on glibc, pango, udev and the DRM subsystem. The latter dependencies are far more likely to have bugs. And even if they weren’t, I am much more interested in patching them than I am in patching readline. So until I acquire the ability to write perfect code on the first try, I will always use kernels with CONFIG_VT=y.
And VTs (+fbcon) don’t depend on the DRM subsystem? kmscon doesn’t work without pango? kmscon requries DRM? Could you please prove your claims, this is not what I see.
I am fine if you think the code behind CONFIG_VT is more reliable than kmscon. I doubt you spend as much time debugging it as I did, but to be honest, I don’t care. I don’t want VTs, so I wrote kmscon. Feel free to disagree and stay with VTs. I have no intention to force anyone to use kmscon.
This is another version of Maciej Piechotka’s question, but in this version of the future, what is the equivalent to the current practice of switching to another console to kill the GUI after it’s crashed? (I’ve had to do this relatively recently on my Ubuntu 12.10 system, so this isn’t a thing of the past.) I certainly hope the answer isn’t the Windowsy procedure of rebooting.
The equivalent ist: Switching to another console, killing the GUI and restarting it.
In other words: Nothing changes. Whether bash is running in the linux console or in kmscon doesn’t matter.
To clarify – Ubuntu is running systemd-logind in Ubuntu 13.10.
What if this assumption is wrong? :
“It is hard to work with such a diverse and moving target, however, they always have something in common: Only one desktop session is active at a time!”
For example, one may have five Wayland sessions, and five seats, each seat wanting to address its own session. Or even ten seats, two per session. Wouldn’t every Wayland session, a lot more than one, be active at the same time? Thanks for the update 🙂
I think the implication is that only one desktop is active *on a particular seat*. There might be multiple active users on the system in a multi-seat case, but each one is only going to have one thing actively controlling the display.
That might be a problem. One desired direction for programs like Steam is to become a wayland client and take over the window management completely. This will resolve all the conflicts and fullscreen issues (pop up messages, chats…), while allowing the user to maintain a second wayland\x a Ctrl+Alt+F8 away.
XBMC too; Running an XBMC session on a monitor and switching away to a window manager for maintenance or whatever would be important for non cli users.
So, I’d definitely want to see more then “Only one desktop session is active at a time!”.
What you want is exactly the setup described in the article. You have to know that “active” means “in foreground”. You can have as many sessions as you want, but only one of them can be _active_. Or how do you think multiple sessions can be active? You cannot share a monitor between sessions at a time.
Switching between sessions will always be possible. So you can switch to XBMC, Weston, Xorg, Steam, console … whenever you want. (btw., steam doesn’t really belong in that least. Session management is at a lower level than that, but according to your description it has a 1-to-1 mapping so we could see it as session).
To clarify, we don’t have global keyboard management currently (with VTs) either, do we?
VTs allow both. In default mode, VTs manage input devices and forward data via the built-in TTY to the applications. As this is highly limited to old-school VT102 like keyboard-handling, sessions like xserver disable it via K_OFF/K_RAW and do keyboard handling themselves. This disables any keyboard handling in the VT, which has the side-effect that VT-switching is now left to the active session.
So with VTs global keyboard management is possible, but undesirable/problematic/unused in practice? Well, it looks like it’s pretty much the same with Logind, so you don’t lose anything compared to the current situation.
We have to be careful not to lump VT session management and VT terminal emulation together. They’re intertwined in kernel space, but should be seen as two different things. While VT terminal emulation in the kernel is horrible and I doubt anyone likes it, VT session management is what is targeted by this proposal.
Keyboard handling+forwarding is a huge part of terminal emulation so beyond the scope of this article. But session-management can also handle keyboard shortcuts to switch between sessions. This we will loose; but we loose only the ability to implement it as in it’s current form, the VT layer doesn’t implement it properly either (due to K_OFF/K_RAW).
Wouldn’t it be possible to put some sort of handling in the kernel to pick up certain key combos to work as a way to hand keyboard control back over to the session manager? Or is that what you were talking about with the magic sysreq mention? (I don’t think actually using the magic sysreq stuff itself is a good plan… It’s awful hard to input that stuff on my laptop, It’d probably be better to use a different set of keys)
Yeah, that’s exactly what I was talking about. We could have a background daemon which just listens for magic shortcuts (of course, can be configured via /etc/whatever.conf) and puts control back to logind in case of emergencies. We could even implement that in logind itself, if we want.
Putting that into the kernel has two advantages:
1) We could drop the input events for the given shortcuts so userspace no longer receives these events (so we don’t have two places where we could react to the same shortcut).
2) It is more likely to work with kdb (hpa made me aware of that, but I haven’t spend much time proving that claim. It does sound reasonable, though).
However, both reasons have little impact on my decision as they also don’t work with the current VT implementation. If someone cares for these things, they’re free to patch the VT layer. I won’t since I don’t care.
Once VTs get kicked out of the kernel, how much work do you think it would take to kick TTYs out too?
TTYs won’t get kicked out. At least I see no reason why. They are pretty useful and the only thing they do is stream management, so they’re fine, I guess.
Fascinating blog! Is your theme custom made or did you download it from somewhere?
A design like yours with a few simple tweeks would
really make my blog stand out. Please let me know where you got your theme.
Kudos
Really good article, thank you!
“During oops/panic the kernel will activate them automatically and notify logind asynchronously via sysfs.”
How can you notify logind after a panic? Isn’t everything halted?
Yeah, on panic we never schedule user-space again, but on oopses we do.