Linux Wii Remote Driver Updates

Linux Bluetooth HID handling has been slightly broken in the kernel since several years. Reconnecting devices caused a kernel oops nearly every time you tried. Two months ago I sat down and rewrote the HIDP session handling and fixed several bugs. The series now got merged and will be part of linux-3.10. If you still encounter bugs please leave me a note.

Based on this I tried recovering my hid-wiimote work. Since the last time I reverse-engineered Nintendo devices, a lot has happened. The most significant change is probably the release of the Wii U. While the Gamepad is still an ongoing target for r/e, other devices were much easier to get working. Motivated by the new hardware I just got, I finally figured out a reliable way for extension hotplugging. This wasn’t supported in the kernel until now and required nasty polling-techniques to work around race-conditions in the proprietary protocol (who designs asynchronous protocols without protocol barriers?). So I rewrote most of the core device handling and moved the input parsers to a module based infrastructure. This allows us to easily extend the hid-wiimote driver to support new devices that are based on the same protocol.

The series is still pending on the linux-input ML but I hope to get basic hotplugging support into linux-3.10. Further support for the built-in speaker device or the Wii U Pro Controller will hopefully follow with 3.11.

The xwiimote user-space stack has already been updated and I will push the changes this weekend.

Wii U Gamepad Connection

I’ve been playing with the Wii-U Gamepad lately and am trying to figure out how it connects to the Wii U console. Once we get that working, we can start reverse-engineering the protocol and write a linux driver for it. It would make a great remote display for every linux box. So how does it work?

The communication between Wii-U and GamePad is done via 5Ghz Wi-Fi. It uses the range 5150-5250 Mhz (Sony UWA-BR100 is a nice dual-band ath9_htc USB dongle with perfect linux support). The console opens a soft AP without any encryption. SSID is set similar to “WiiU34af2c5fa6134af2c5fa61c_STA1″. The GamePad connects to this AP and then creates some private link. I haven’t figured out how this works, yet.

The IE fields do not advertise any Wifi-Direct (P2P), Wifi-Display (WFD) or Direct-Link (TDLS) features. The only features found are WMM QoS fields.

How to proceed? I need to figure out how to create a soft-AP with the advertised features so I can make the GamePad connect to me. Two Nintendo extensions are advertised “OUI a4:c0:e1″ which probably identify the AP. The other vendor IEs are Broadcom/EPIGRAM IDs which can also be found on other networks. After that, I need to test P2P discovery, TDLS discovery or 802.11e DLS setup to find out what kind of direct-link Nintendo uses. According to Broadcom’s Dino Bekis a form a Miracast is used which would mandate P2P or TDLS.

If anyone has more information on that, I’d be very thankful!

Btw., dhcp is provided on the unprotected soft-AP and I can ping the console but my port-scans didn’t return any useful information. I will try connecting to WFD/RTSP default port 7236 next…

 

Soft-AP during Synchronization:

BSS 34:af:2c:5f:a6:1c (on wlan1)
	TSF: 3126337 usec (0d, 00:00:03)
	freq: 5180
	beacon interval: 100
	capability: ESS (0x0001)
	signal: -55.00 dBm
	last seen: 3208 ms ago
	Information elements from Probe Response frame:
	SSID: WiiU34af2c5fa6134af2c5fa61c_STA1
	Supported rates: 6.0* 9.0 12.0* 18.0 24.0* 36.0 48.0 54.0 
	HT capabilities:
		Capabilities: 0x1c
			HT20
			SM Power Save disabled
			RX Greenfield
			No RX STBC
			Max AMSDU length: 3839 bytes
			No DSSS/CCK HT40
		Maximum RX AMPDU length 16383 bytes (exponent: 0x001)
		Minimum RX AMPDU time spacing: 8 usec (0x06)
		HT RX MCS rate indexes supported: 0-15
		HT TX MCS rate indexes are undefined
	HT operation:
		 * primary channel: 36
		 * secondary channel offset: no secondary
		 * STA channel width: 20 MHz
		 * RIFS: 1
		 * HT protection: no
		 * non-GF present: 0
		 * OBSS non-GF present: 0
		 * dual beacon: 0
		 * dual CTS protection: 0
		 * STBC beacon: 0
		 * L-SIG TXOP Prot: 0
		 * PCO active: 0
		 * PCO phase: 0
	Vendor specific: OUI a4:c0:e1, data: f5 00
	Vendor specific: OUI a4:c0:e1, data: f4 10 4a 00 01 10 10 44 00 01 02 10 41 00 01 01 10 12 00 02 00 00 10 53 00 02 00 84 10 3b 00 01 03 10 47 00 10 22 21 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 21 00 08 42 72 6f 61 64 63 6f 6d 10 23 00 06 53 6f 66 74 41 50 10 24 00 01 30 10 42 00 01 30 10 54 00 08 00 06 a4 c0 e1 f4 00 01 10 11 00 10 57 69 69 55 33 34 61 66 32 63 35 66 61 36 31 63 10 08 00 02 00 84
	Vendor specific: OUI 00:10:18, data: 02 00 00 04 00 00
	WMM:	 * Parameter version 1
		 * u-APSD
		 * BE: CW 15-31, AIFSN 2, TXOP 1504 usec
		 * BK: CW 15-1023, AIFSN 7
		 * VI: CW 15-31, AIFSN 3, TXOP 3008 usec
		 * VO: CW 15-31, AIFSN 3, TXOP 1504 usec

Soft-AP during normal operation with GamePad:

BSS 34:af:2c:5f:a6:1c (on wlan1)
	TSF: 413388858 usec (0d, 00:06:53)
	freq: 5180
	beacon interval: 100
	capability: ESS Privacy (0x0011)
	signal: -48.00 dBm
	last seen: 3201 ms ago
	Information elements from Probe Response frame:
	SSID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
	Supported rates: 6.0* 9.0 12.0* 18.0 24.0* 36.0 48.0 54.0 
	TIM: DTIM Count 1 DTIM Period 3 Bitmap Control 0x0 Bitmap[0] 0x0
	RSN:	 * Version: 1
		 * Group cipher: a4-c0-e1:4
		 * Pairwise ciphers: a4-c0-e1:4
		 * Authentication suites: a4-c0-e1:2
		 * Capabilities: 4-PTKSA-RC (0x0008)
	HT capabilities:
		Capabilities: 0x1c
			HT20
			SM Power Save disabled
			RX Greenfield
			No RX STBC
			Max AMSDU length: 3839 bytes
			No DSSS/CCK HT40
		Maximum RX AMPDU length 16383 bytes (exponent: 0x001)
		Minimum RX AMPDU time spacing: 8 usec (0x06)
		HT RX MCS rate indexes supported: 0-15
		HT TX MCS rate indexes are undefined
	HT operation:
		 * primary channel: 36
		 * secondary channel offset: no secondary
		 * STA channel width: 20 MHz
		 * RIFS: 1
		 * HT protection: no
		 * non-GF present: 1
		 * OBSS non-GF present: 0
		 * dual beacon: 0
		 * dual CTS protection: 0
		 * STBC beacon: 0
		 * L-SIG TXOP Prot: 0
		 * PCO active: 0
		 * PCO phase: 0
	Vendor specific: OUI 00:90:4c, data: 07 00 45 55 17
	Vendor specific: OUI 00:10:18, data: 02 01 00 04 00 00
	WMM:	 * Parameter version 1
		 * u-APSD
		 * BE: CW 15-31, AIFSN 2, TXOP 1504 usec
		 * BK: CW 15-1023, AIFSN 7
		 * VI: CW 15-31, AIFSN 3, TXOP 3008 usec
		 * VO: CW 15-31, AIFSN 3, TXOP 1504 usec

Linking binary data

Took me half a weak to figure this out, but here is a quick description how to link your binary data into a library/executable:

We want this to:

  • work with GNU-ld
  • work with GNU-gold
  • work with GNU-libtool (if required)
  • work with cross-compiling
  • work with LLVM
  • not require any external non-standard tools

For now lets assume we have a file src/mydata.bin that contains your binary data that you want to link into your executable as a huge C-array.

With GNU-autotools

If you use GNU-autotools, add the following rule to your Makefile.am:

CLEANFILES += src/*.bin.lo src/*.bin.o

src/%.bin.lo: src/%.bin
     $(AM_V_GEN)$(LD) -r -o "src/$*.bin.o" -z noexecstack --format=binary "$<"
     $(AM_V_at)$(OBJCOPY) --rename-section .data=.rodata,alloc,load,readonly,data,contents "src/$*.bin.o
     $(AM_V_at)echo "# $@ - a libtool object file" >"$@"
     $(AM_V_at)echo "# Generated by $(shell $(LIBTOOL) --version | head -n 1)" >>"$@"
     $(AM_V_at)echo "#" >>"$@"
     $(AM_V_at)echo "# Please DO NOT delete this file!" >>"$@"
     $(AM_V_at)echo "# It is necessary for linking the library." >>"$@"
     $(AM_V_at)echo >>"$@"
     $(AM_V_at)echo "# Name of the PIC object." >>"$@"
     $(AM_V_at)echo "pic_object='$*.bin.o'" >>"$@"
     $(AM_V_at)echo >>"$@"
     $(AM_V_at)echo "# Name of the non-PIC object" >>"$@"
     $(AM_V_at)echo "non_pic_object='$*.bin.o'" >>"$@"
     $(AM_V_at)echo >>"$@"

Now you can simply use src/mydata.bin.lo as libtool-object file in any _LIBADD automake variable. This rule causes $(LD) to link the binary into the object file src/mydata.bin.o (via partial-linking/relocatables). After that we create the corresponding libtool object file src/mydata.bin.lo. Note that the comments in the libtool file are mandatory. You must not remove them!

The objcopy line causes the .data section to be renamed to .rodata and marked as read-only. You can drop it if you require write access.

Without GNU-autotools

Obviously, this is much easier. You can simply use:

$(LD) -r -o "src/mydata.bin.o" -z noexecstack --format=binary "src/mydata.bin"
$(OBJCOPY) --rename-section .data=.rodata,alloc,load,readonly,data,contents "src/mydata.bin.o"

This produces src/mydata.bin.o which you can then link like any other object.

That’s already it!

You can now access the binary data via:

extern const char _binary_<path>_start[];
extern const char _binary_<path>_end[];

In our case this would be:

extern const char _binary_src_mydata_bin_start[];
extern const char _binary_src_mydata_bin_end[];

The linker automatically creates these symbols. It also creates *_size symbols but I never had to use these. The linker also alignes the start address correctly, but the content isn’t touched, of course.

The data is placed in .data by default and marked as read/write. The objcopy command renames it to .rodata and marks it as read-only. Drop it if you require write access.

If you think you know another way to do this, please let me know! I have tried lots of other ways and they all fail in some subtle way:

  • Converting it into a C-source file with a static array. Problem: It works but the C-compiler might take considerably longer to compile the sources. On my machine a 2MB array file took >10min to compile.
  • Using objcopy to convert binary files into object files. Problem: This requires the target architecture as -B command. Unfortunately, it has some weird format and doesn’t accept the standard GNU $host_cpu kind of strings. For instance it requires i386:x86-64 instead of x86_64. I haven’t found a way to get these BFD strings reliably via autoconf.
  • Use objcopy without -B to produce architecture=UNKNOWN object files and link via –accept-unknown-input-arch with GNU-ldProblem: This doesn’t work with GNU-gold as it doesn’t support this command-line option.
  • Avoid partial-linking and add this to your LDFLAGS: -Wl,--format=binary -Wl,src/mydata.bin -Wl,--format=default Problem: This marks the file as requiring an executable-stack. You can fix this with -Wl,-z,noexecstack but that forces all other linked files to also not require an executable stack. And also, GNU-gold doesn’t support --format=default, it requires --format=elf which itself isn’t supported by GNU-ld. Hilarious, right?

I also got some other hints to use GNU-as or other nice workarounds. However, I wanted a solution that uses the tools that were created for that task. I mean, why should I create an assembler or C source file to link binary data? Seriously, the only reason to do this is to work around crappy binutils interfaces. I want ld and objcopy to do the task and if they can’t, then we should fix them!

Advanced DRM Mode-Setting API

I recently wrote a short How-To that introduces the linux DRM Mode-Setting API. It didn’t use any advanced techniques but I got several responses that it is a great introduction if you want to get started with linux DRM Mode-Setting. So I decided to go further and extend the examples to use double-buffering and vsync’ed page-flips.

The first extension that I wrote can be found here:

https://github.com/dvdhrm/docs/blob/master/drm-howto/modeset-double-buffered.c

It extends the old example to use two buffers so we no longer render into the front-buffer. It reduces a lot of tearing that you get when using the single-buffered example. However, it is still not perfect as you might swap the front and back buffer during a scanout-period and the display-controller will use the new buffer in the middle of the screen. So I extended this further to do the page-flip during a vertical-blank period using drmModePageFlip(). You can find this example here:

https://github.com/dvdhrm/docs/blob/master/drm-howto/modeset-vsync.c

This example also shows how to wait for page-flip events and integrate it into any select(), poll() or epoll based event-loop. Everything regarding page-flip-timing beyond that point depends on the use-cases and can get very hard to get right. I recommend reading Owen Taylor’s posts #1 and #2 on frame-timing for compositors.

There are still many more things like “DRM hardware-accelerated rendering”, “DRM planes/overlays/sprites”, “DRM flink/dmabuf buffer-passing” that I want to write How-Tos for. But time is short around Christmas so that’ll have to wait until next year.

Feedback is always welcome and you can find my email address in all the examples. Happy reading!

KMSCON Introduction

KMSCON is a KMS/DRM-based system console with an integrated terminal emulator. It was designed to replace the linux-kernel-console and virtual terminals (VTs). When run in default-mode, KMSCON allocates a virtual terminal and provides a terminal-emulator on it. It can thus be used as drop-in replacement for the linux-console and agetty. Compared to the linux-console, KMSCON provides a rich set of enhanced features including full-internationalized keyboard handling, full UTF8 input/font support, hardware-accelerated rendering, multi-seat support and more.

If run in listen-mode, KMSCON can also replace virtual terminals. This allows to run multiple graphics-servers (like kmscon, X-Server, Wayland Compositors) simultaneously on all seats not only on the default seat seat0.

Parts of this article expect the reader to be familiar with the term multi-seat. KMSCON runs on single-seat and multi-seat setups, but to understand the terms used here, you should read up on Wikipedia. On linux, the default seat is called seat0. This is also the primary seat for single-seat setups or if KMSCON is compiled without multi-seat support.

Default Mode

In default-mode KMSCON opens a virtual terminal (VT) to provide a terminal-emulator. It first checks whether the controlling terminal is a VT, if it’s not it tries to find an unused VT via VT_OPENQRY. You can also specify a VT on the command line via --vt=/dev/tty5, or –vt=tty5, or –vt=5. This syntax allows backwards compatibility to most getty implementations.

After startup the VT is automatically activated and switched to (this can be prevented with –no-switchvt). You can use the standard linux keyboard shortcuts ctrl+alt+F1 to ctrl+alt+F12 to switch between your VTs. KMSCON integrates seamlessly into existing setups and can be used in parallel with X-Servers, Wayland-Compositors or the linux-console. Only one VT is occupied by KMSCON, so you can still use the linux-console on all other VTs. But you can also run a KMSCON process on each VT replacing the linux-console everywhere.

By default, KMSCON spawns /bin/login on the new terminal session. You can change this with the –login option. However, for security reasons it is recommended to keep the default. If a session exits, it is automatically restarted. To terminate KMSCON send SIGTERM to the main process or use a daemon manager like systemd.

KMSCON tries to be backwards-compatible in terms of user-interaction to the linux-console. However, the internal terminal-emulator is developed to be compatible with xterm. This is because the linux-console‘s terminal-emulation layer is very limited and incompatible to many other terminal-emulators. On the other hand, xterm is the de-facto standard in terminal-emulation under linux and provides many enhanced control-sequences. Therefore, KMSCON tries to behave exactly like xterm does. This isn’t always as easy as it sounds so please file bug-reports if you encounter any incompatibilities.

With this knowledge we can now replace agetty. So instead of:

  /sbin/agetty --noclear tty5 38400 linux

we now run:

  /usr/bin/kmscon --vt=tty5 --no-switchvt

to replace the linux-console with KMSCON on VT-5.  KMSCON also ships optional systemd service files in ./kmscon/docs/*.service

Keyboard Input

KMSCON uses libxkbcommon for keyboard control. This library implements huge parts of the X11 Keyboard Specification without any dependencies to X11 source-code. It was developed to allow other applications than X-Server to provide internationalized keyboard handling. It recently saw its first release after several years of development but is currently not available in all major linux distributions. However, many distributions already provide experimental packages for it.

With XKB support on board, KMSCON can provide the same keyboard handling as the X-Server does. That is, keyboards are configured via the RMLVO options (Rules Model Layout Variant Options). The most commonly used option is probably –xkb-layout=<language> to change the keyboard layout. All other options are mostly unused.

Any keyboard setups that can be used with X11 can also be used with xkbcommon. Simply create your X11 keyboard layouts or use one of the many existing ones and configure KMSCON to use it.

Video Devices

KMSCON got its name from the linux kernel Direct Rendering Manager (DRM) subsystem (not to be confused with DRM: Digital Rights Management). DRM drivers provide a common set of APIs to perform Mode-Setting, called Kernel Mode-Setting (KMS). This is the API that is used by X-Server to program video output. KMSCON uses the same API and thus provides mostly the same compatibility to GPU drivers as X-Server does.

But KMSCON also supports linux fbdev devices for backwards-compatibility. In fact, the modular interface allows to provide drivers for all kinds of video hardware. However, it is recommended to write DRM drivers for your video-hardware instead of writing user-space drivers for KMSCON or X-Server. Because this allows all user-space programs to use the DRM API to access any type of video output device without extra user-space support for each driver.

Right from the beginning KMSCON provided multiple-GPU support. This means, KMSCON automatically picks up all connected GPUs and provides a system console on them. You can even hotplug new DisplayLink GPUs and KMSCON detects them during runtime and instantly provides a console on them. Unfortunately, GPU detection isn’t as straightforward as one would think. On some systems there are GPUs that shouldn’t be touched by KMSCON. That’s why KMSCON provides the –gpu switch to change the GPU selection algorithm. A GPU blacklist configuration file or option is also on its way.

On fbdev and dumb-DRM devices, KMSCON uses 2D rendering without any hardware acceleration. But if KMSCON is compiled with full DRM video backends, it can use mesa3D’s OpenGLESv2 implementation to provide hardware-accelerated rendering. Use –hwaccel to enable this during runtime. It can speed up rendering by multiple orders of magnitude. However, on older systems it might even slow down rendering. It is disabled by default and needs to be explicitly enabled with –hwaccel.

Seat Selection

By default, KMSCON runs on the seat that it was started on. At most times, this is the default seat seat0. However, you can run KMSCON on other seats with the –seats option. It takes a list of seat names that KMSCON will run on. Each seat is independent and you can either run one KMSCON process for each seat or you can even run a single KMSCON process on multiple seats. The latter will allow sharing of system-resources like font glyph-caches. However, if 3D-hardware-acceleration is enabled, unexpected latencies may occur in the DRM drivers and you shouldn’t share a single KMSCON process across multiple seats for decent user-experience.

On seat0 KMSCON can make use of VTs. However, on seats other than seat0 (or even on seat0 if VTs are disabled) linux does not provide VTs. Therefore, KMSCON has to run without them. This means, KMSCON will automatically activate itself and run unconditionally on this seat. Unfortunately this has the side-effect that no other graphics-server (like X-Server) can run on this seat. Without the synchronization API of VTs there is no way to dispatch the graphics/input devices between multiple graphics servers. Therefore, most users will prefer running an X-Server on those seats. However, if no X11 is needed, KMSCON runs perfectly fine on those seats, too.

Listen-Mode

In default-mode KMSCON runs only on seats that are available during startup and terminates after all seats it runs on are either gone or their controlling VT hung up. This is perfect for VTs on seat0 but it requires another process to start KMSCON for each new seat showing up.

In listen-mode KMSCON runs as daemon waiting for new seats to show up. It automatically spawns a new terminal session on each seat matching the –seats option. KMSCON will not exit until SIGTERM is caught. Note that KMSCON ignores seats with VTs (normally only seat0) in this mode as you should run KMSCON in default-mode on those seats.

VT Replacement

The linux-kernel-console and VTs are tightly bound to each other. There is only one kernel configuration option to control them both: CONFIG_VT. KMSCON can already replace the linux-kernel-console but to disable CONFIG_VT, we also need a proper replacement for virtual terminals. As previously noted, KMSCON can run smoothly without VTs, however, this means that you can only run a single graphics-server on a seat and most users will probably choose X-Server here (and I would do so, too).

Fortunately, KMSCON already provides a solution to this. The –cdev-session option provides fake VTs for all seats that do not have real VTs. This allows to run X-Server, KMSCON, a Wayland Compositor and any other graphics-server simultaneously without changing a single line of code. However, this is beyond the scope of this introduction. Stay tuned for a follow-up post that introduces KMSCON session support and cdev-sessions in more detail.

Current State

KMSCON works! No special setup, no special system requirements and no weird dependencies are needed. You can check out KMSCON on any major linux distribution. Right now. What you need is:

  • libxkbcommon: This library has no external dependencies and can easily be installed on any distribution. Many distributions even provide experimental packages for it. See xkbcommon.org.
  • libudev: Udev is used for hotplugging and device detection. Every major distribution provides it.
  • kmscon: You can get KMSCON from github.

Everything else is optional! For proper font-rendering you also need pango or freetype2. For DRM video output you need libdrm. For 3D-hardware-acceleration mesa3D is needed. For multi-seat support systemd-logind is used.

However, please note that KMSCON is still experimental. I do my best to keep backwards-compatibility and fix all bugs that are reported. But I am unable to test KMSCON on all imaginable setups. So please report all bugs to github.com/dvdhrm/kmscon and I promise to have a look at it.

Furthermore, the TODO list grows steadily and I still have lots of ideas how to improve KMSCON. New ideas are very welcome, but please bear with me if it takes more time than expected to implement them. There is probably lots of stuff with higher priority on my list. But patches are highly appreciated even if they implement low-priority features.

There is a man-page kmscon(1) that explains many concepts and command-line arguments right-away. But do not hesitate to contact me personally if you have any questions about KMSCON. And for all developers I am doing a presentation about deprecating CONFIG_VT and KMSCON during FOSDEM-13 in Brussels next year. Hope to see you there!

Linux DRM Mode-Setting API

The Direct Rendering Manager (DRM) is a subsystem of the linux kernel that manages access to graphics cards (GPUs). It is the main video API used by X.org‘s xserver and the xf86-video-* video drivers. However, it can also be used by independent programs to program video output without using the xserver or wayland. In the past most other projects used the much older fbdev API, however, with more and more drivers being added to the DRM subsystem, there is really no reason to avoid DRM on modern computers, anymore. Unfortunately, there hasn’t been any documentation of the DRM API, yet.

DRM-Modesetting HowTo

I have written a short introduction into the DRM mode-setting API, which can be found on github. It is a full C-file with detailed comments on what is needed to perform simple mode-setting with the DRM-API. I embedded the documentation directly into the source file as this makes reading a lot more convenient:

https://github.com/dvdhrm/docs/blob/master/drm-howto/modeset.c

This document does not describe the whole DRM API. There are parts like the OpenGL-rendering-pipeline, which are driver-dependent and which should almost never be accessed outside of the mesa-3D implementation. Instead, this document describes the API which is needed to write simple applications performing software-rendering similar to fbdev but with the DRM API.

Furthermore, this document is not free of errors. So please contact me if something is wrong, if essential parts are missing or if you intend to extend this documentation.

More tutorials will follow, including “DRM double/triple-buffering”, “DRM vsync’ed pageflips”, “DRM hardware-accelerated rendering”, “DRM planes/overlays/sprites” and more.

I hope you enjoy this short introduction.

Edit: For a new series of How-To’s, see http://dvdhrm.wordpress.com/2012/12/21/advanced-drm-mode-setting-api/

Deprecating CONFIG_VT

CONFIG_VT is the kernel configuration option for the VT subsystem of the linux kernel. It enables the in-kernel VT102 emulator which is also known as “linux kernel console” or “VT-1 to VT-6″ or “<ctrl>+<alt>+<F1-F6>” or whatever you call it. It has been in the kernel since the beginning and people started integrating it into all parts of the system. This makes it much harder to remove it. But lets first look at the reasons why I think CONFIG_VT should be removed (or shorter: CONFIG_VT=n):

  • No multi-seat capabilities: The VT system is highly incompatible with multi-seat environments. Current multi-seat implementations simply bind all VTs to the default-seat (or “seat0″) and all other seats just ignore all VTs. Therefore, you can think of all other seats already running with CONFIG_VT=n.
  • No Unicode support: The linux console has very limited Unicode support. It can support UTF-8, but the fonts are too limited to even display all European languages (don’t let me even start talking about CJK…). This definitely needs to be replaced by a more internationalized console.
  • No internationalized keyboards: Similar to the bad Unicode support the keyboard-handling is bad, too. Xkb provides many more features that are needed to support fully internationalized keyboards. This definitely needs to be improved, too.
  • No Hardware-accelerated rendering: The linux console draws everything via simple 2D blitting operations directly into the framebuffer. This has been optimized with partial-redraws to improve performance. However, running linux on a slower machine with many monitors will horribly slow down your system. User-space can take advantage of OpenGL to draw consoles much faster.
  • Limited modesetting support: Modern graphics cards can run multiple monitors simultaneously. With the “DisplayLink” technology you can even add more via USB. The linux-console runs once per framebuffer and does not provide any mode-setting configuration. With the DRM API user-space can provide all this out-of-the-box.
  • Limited VT102 support: xterm, gnome-terminal, konsole, xfce-terminal, … all support much more than the limited VT102 functionality. The linux console does not even support the full VT100 specs. Moreover, it introduced many control-sequences that conflict with known control-sequences from other terminal emulators. Dropping the linux-console will help reducing conflicts between terminal-emulators.
  • Misdesigned VT API: This is kind of a personal issue, but I never liked the VT API. Synchronization is done via signals, acknowledgement requests are used and overall, it’s horrible to work with. The VT allocation/deallocation-logic is a mess and I never want to work with it again.
  • No AA-fonts: Anti-aliasing can enhance readability a lot even though some people might disable it for the same reasons. But linux is all about choice so lets support it.
  • Many more…

Some of the issues can be solved by improving the existing code. But this is not the solution as no-one wants the stuff in the kernel. Think of Xkb, pango, freetype2, mesa, etc. being in the kernel. No way that will be done. Terminal emulators might not be seen as GUI, but I think of them as UI. Hence, why would the kernel implement an UI? UIs belong to user-space, so do all terminal-emulators.

The curious reader might have noticed that many of the points are related to the linux-console, but the graphical VT API is independent of it. Technically, it would be much work splitting them off as they are tight strongly together in the kernel code. But many points also apply to the VT-API so getting rid of all of it seems to be the right way to me.

Replacements

First of all, is there really a need for replacement? Graphical environments provide many terminal emulators and remote devices can be controlled via ssh. So I can only think of 2 obvious reasons why one wants the linux-console:

  1. Emergency console: In the past the xorg-server was often known to be buggy with new or fancy hardware. I think that this has improved considerably over the last years, but some might not agree. A safe fallback like the linux-console is nice to have to recover a system without rebooting. Or to fix your graphical-setup when your xorg-server won’t start up. However, please take into account that the kernel-console has no advantages over a console implemented in user-space. In fact, without a working user-space you will not be able to do anything with the kernel-console, either. It may be able to display information, but you cannot interact with it as your shell runs in user-space and every command that you execute runs in user-space, too.
  2. Lightweight UI: The xorg-server is heavy. It has lots of dependencies and many terminal emulators pull in a lot more. Even though I think that this can be reduced to a very lightweight xorg-server, the linux-console comes without any dependencies which is really nice for smaller systems and debugging.

I think both points are quite important but aren’t tightly bound CONFIG_VT. I recently started working on kmscon, a terminal-emulator implemented in user-space which integrates perfectly well with multi-seat and/or non-graphical environments. It can be built without any dependencies but has optional support for all the feature mentioned above including hardware-accelerated rendering and full internationalization support. It serves on my machines as emergency-console and replaces the linux-console. It is also very lightweight if built without all the optional extensions, but still provides more features than the linux console. It is still experimental, though.

Then there is also fblog. It is a kernel driver which I recently posted to the LKML for inclusion into mainline. It is still under development but is a nice kernel-log-display and replacement for fbcon. It cannot be used with CONFIG_VT enabled as it conflicts with the kernel-console and provides a subset of the features. When enabled, fblog prints the in-kernel log-buffer onto all connected framebuffers. This allows debugging of kernel errors, kernel boot, kernel panics, kernel oopses and more when user-space failed. Without CONFIG_VT there is no driver displaying this information and one would have to use a serial-console or similar. fblog comes in handy by displaying this information to all connected framebuffers. It can be enabled during boot/shutdown and when requested. It can be disabled when starting a graphical-environment and re-enabled when stopping it.

Problems

There are still problems when removing CONFIG_VT. If you want to run two applications (lets say kmscon and xorg-server) simultaneously on one seat, how does one application know when it is active and when to go into background. This was done with the VT-API but it is no longer available. Of-course, one could easily replace it with something newer, but many people think that this is not needed. Instead we should avoid the VT-logic entirely. Probably, we can use a system-compositor which is lightweight and synchronizes access to the graphics-devices.

I personally use another approach. I have a manager-daemon running in background which listens to keyboard-input. kmscon is automatically started on boot (but you could also easily make xorg-server the default) and if I press ctrl+mod4+F12 kmscon is notified by the manager-daemon to go into background and xorg-server is started. The xorg-server doesn’t include such a notification API and is very fragile when other applications access the graphics hardware at the same time. Therefore, I simply stop xorg-server and press ctrl+mod4+F12 again to get kmscon into foreground. Pressing ctrl+mod4+F12 twice very fast will attempt to kill all running xorg-servers or other graphical applications except kmscon and then push kmscon into foreground. This is some kind of emergency-helper when closing xorg-server is not possible.

System-Compositor

The approach by the Wayland people is somewhat different. Without going into much detail about Wayland, you need to know that the Wayland API was designed to be stackable without any overhead. Or nearly no overhead. So you can run a system-compositor which has master-access to the graphics devices. All clients run full-screen and the windowing functionality is very limited. If you start kmscon, it should simply connect to the system-compositor and talk to the graphics-devices via the Wayland protocol. The same way should an xorg-server or normal Wayland-compositor (like Weston) connect to the system-compositor. Because every client is full-screen on the system-compositor it can simply forward the scan-out buffer to the hardware without copying any data. Hence, the system-compositor can be very lightweight and is super fast.

However, such an system-compositor is not yet available. The Weston compositor is still under heavy development and therefore, this is something where much work is still to do. But it looks like it will be the best replacement for CONFIG_VT and hopefully won’t take too long.

Outlook

I think CONFIG_VT is dead. It hasn’t been updated for long and many better ideas have emerged. Many low-level projects are working towards making it obsolete and I hope I can continue contributing to this with fblog and kmscon. However, due to it’s tight integration into the whole system, it will be a hard task to replace CONFIG_VT entirely. As a first step it was already made optional by many applications due to multi-seat support. But the following work is much harder. Finding a good replacement which makes everyone happy. It will be interesting how people will accept it. Considering the hostility against the big changes in user-space that we have seen recently, it will be fun how people loose themselves to fight for their beloved linux-console. But maybe everyone is just happy about killing off CONFIG_VT. Lets hope for the best!