On Wifi-Display, Democratic Republics and Miracles

For a long time connecting your TV or other external monitors to your laptop has always been a hassle with cables and drivers. Your TV had an HDMI-port, but your laptop only DVI, your projector requires VGA and your MacBook can only do DP. Once you got a matching adapter, you noticed that the cable is far to short for you to sit comfortably on your couch. Fortunately, we’re about to put an end to this. At least, that was my impression when I first heard of Miracast.

Miracast is a certification program of the Wifi-Alliance based on their Wifi-Display specification. It defines a protocol to connect external monitors via wifi to your device. A rough description would be “HDMI over Wifi” and in fact the setup is modeled closely to the HDMI standard. Unlike existing video/audio-streaming protocols, Miracast was explicitly designed for this use-case and serves it very well. When I first heard of it about one year ago, I was quite amazed that it took us until late 2012 to come up with such a simple idea. And it didn’t take me long until I started looking into it.

For 4 months now I have been hacking on wifi, gstreamer and the kernel to get a working prototype. Despite many setbacks, I didn’t loose my fascination for this technology. I did complain a lot about the hundreds of kernel dead-locks, panics and crashes during development and the never-ending list of broken wifi-devices. However, to be fair I got the same amount of crashes and bugs on Android and competing proprietary Miracast products. I found a lot of other projects with the same goal, but the majority was closed-source, driver-specific, only a proof-of-concept or limited to sink-side. Nevertheless, I continued pushing code to the OpenWFD repository and registered as speaker for FOSDEM-2014 to present my results. Unfortunately, it took me until 1 week before FOSDEM to get the wifi-P2P link working realiably with an upstream driver. So during these 7 days I hacked up the protocol and streaming part and luckily got a working demo just in time. I never dared pushing that code to the public repository, though (among others it contains my birthday, which is surprisingly page-aligned, as magic mmap offset) and I doubt it will work on any setup but mine..

The first day after FOSDEM I trashed OpenWFD and decided to start all over again writing a properly integrated solution based on my past experience. Remembering the advice of a friend (“Projects with ‘Open’ in their name are just like countries named ‘Democratic Republic of …’”) I also looked for a new name. Given the fact that it requires a miracle to get Wifi-P2P working, it didn’t take me long to come up with something I liked. I hereby proudly announce: MiracleCast

MiracleCast

The core of MiracleCast is a daemon called miracled. It runs as system-daemon and manages local links, performs peer-discovery on request and handles protocol encoding and parsing. The daemon is independent of desktop environments or data transports and serves as local authority for all display-streaming protocols. While Miracast will stay the main target, other competing technologies like Chromecast, Airtame and AirPlay can be integrated quite easily.

The miraclectl command-line tool can be used to control the daemon. It can create new connections, modify parameters or destroy them again. It also supports an interactive mode where it displays events from miracled when they arrive, displays incoming connection attempts and allows direct user-input to accept or reject any requests.

miracled and miraclectl communicate via DBus so you can replace the command-line interface with graphical helpers. The main objects on this API are Links and Peers. Links describe local interfaces that are used to communicate with remote devices. This includes wifi-devices, but also virtual links like your local IP-Network. Peers are remote devices that were discovered on a local link. miracled hides the transport-type of each peer so you can use streaming protocols on-top of any available link-type (given the remote side supports the same). Therefore, we’re not limited to Wifi-P2P, but can use Ethernet, Bluetooth, AP-based Wifi and basically any other transport with an IP layer on top. This is especially handy for testing and development.

Local processes can now register Sources or Sinks with miracled. These describe capabilities that are advertised on the available links. Remote devices can then initiate a connection to your sink or, if you’re a source, you can connect to a remote sink. miracled implements the transmission protocol and hides it behind a DBus interface. It routes traffic from remote devices to the correct local process and vice versa. Regardless of whether the connection uses Miracast, Chromecast or plain RTSP, the same API is used.

Current Status

The main target still is Miracast! While all the APIs are independent of the protocol and transport layer, I want to get Miracast working as it is a nice way to test interoperability with Android. And for Miracast, we first need Wifi-P2P working. Therefore, that’s what I started with. The current miracled daemon implements a fully working Wifi-P2P user-space based on wpa_supplicant. Everyone interested is welcome to give it a try!

Everything on top, including the actual video-streaming is highly experimental and still hacked on. But if you hacked on Miracast, you know that the link-layer is the hard part. So after 1 week of vacation (that is, 1 week hacking on systemd!) I will return to MiracleCast and tackle local sinks.

As I have a lot more information than I could possible summarize here, I will try to keep some dummy-documentation on the wiki until I the first project release. Please feel free to contact me if you have any questions. Check out the git-repository if you want to see how it works! Note that MiracleCast is focused on proper desktop integration instead of fast prototyping, so please bear with me if API design takes some time. I’d really appreciate help on making Wifi-P2P work with as many devices as possible before we start spending all our efforts on the upper streaming layers.

During the next weeks, I will post some articles that explain step by step how to get Wifi-P2P working, how you can get some Miracast protoypes working and what competing technologies and implementations are available out there. I hope this way I can spread my fascination for Miracast and provide it to as many people as possible!

About these ads

38 thoughts on “On Wifi-Display, Democratic Republics and Miracles

    1. David Herrmann Post author

      libwfd is available here: http://cgit.freedesktop.org/~dvdhrm/libwfd
      I am still working on all the documentation, but given the fact that neither API is stable, I’d like to first get the initial releases ready before doing thorough documentation.

      Regarding systemd-209: I run systemd-208 myself. All I need from systemd-209 is the really awesome sd-bus library (seriously, so much nicer to use than libdbus1!). So what I do is:

      git clone git://anongit.freedesktop.org/systemd/systemd systemd-git
      cd systemd-git
      ./autogen.sh
      ./configure [... the listed options ...]
      make
      sudo cp .libs/libsystemd.* /usr/local/lib/
      sudo cp -r src/systemd /usr/local/include/

      And that’s it! I will include that in the HOWTO that I’m working on.

      Reply
      1. Pavan Kishore

        Hi David,

        I compiled systemd-216 on Ubuntu 12.04 and followed the steps that you mentioned above. However I’m still getting the include errors as below,

        $ make
        make –no-print-directory all-recursive
        Making all in .
        CC src/wifi/miracle_wifid-wifid.o
        In file included from src/wifi/wifid.c:30:0:
        /usr/local/include/systemd/sd-bus.h:31:19: fatal error: memfd.h: No such file or directory
        compilation terminated.
        make[2]: *** [src/wifi/miracle_wifid-wifid.o] Error 1
        make[1]: *** [all-recursive] Error 1
        make: *** [all] Error 2

        -Pavan Kishore

      2. David Herrmann Post author

        You probably need linux-headers >= 3.17 (or kernel-headers or whatever it is called. /usr/include/linux/memfd.h must be available, which it is since 3.17).

  1. Tom Gundersen

    Awesome stuff David!

    Regarding your miracle-dhcp, I’m working on adding adding a sd-dhcp-server lib that you may want to use in the future (and feel free to ping me about the sd-dhcp-client api if you want something it is lacking).

    As to the calling out to ‘ip’, did you have a look at using sd-rtnl to do these things instead? Should be pretty straight-forward (apart from the fact that it is not a public library yet, so you have to do the same hack is with sd-bus).

    Reply
    1. David Herrmann Post author

      Oh, you found the largest hack in the code-base.. damnit! Btw., the comments mention that I will switch to sd-dhcp once you support server-side dhcp. But until then, I don’t want a split code-base so I’ll keep the current helper. And yes, I also want to avoid calling out to ‘ip’, but rtnl is not really my strength so I might query you about that afterwards. Regarding sd-dhcp-client, I will have a look on it anyway. Final reviews tomorrow before 209 will get released, I guess.

      Reply
  2. Platomaniac

    Hi David,

    Hallo from TU Kaiserslautern. We are few guys working on some interactive remote streaming applications(Gaming is working currently,…and every app that involves user interactions are our target). Wanted to have a chat with you about our next idea and prototypes if you have little time for us….our current prototype is pretty much like the CES2014 wireless gaming demo of NVIDIA (http://www.youtube.com/watch?v=RW9WzKUI3n4). Just it is not as polished…but we are doing little less than 60 frames..at this point of our development what we are planning as a future step require little suggestion from you. Would like to write you an email or a skype chat..whatever you is possible for you.
    Looking forward to hear from you..

    Plato

    Reply
  3. Przemo

    I managed to compile miracle using provided above systemd hack. I also had to use ldconfig to get access to systemd libs in /usr/local/libs (fedora 20), but I’m stuck on this:
    bash-4.2$ miracled
    ERROR: dbus: cannot claim org.freedesktop.miracle bus-name: -13 (manager_dbus_connect() in src/miracled-dbus.c:833)

    I missed something or I should make a dbus conf file for miracled?

    Reply
  4. Bruno

    Is there a list of working wifi cards somewhere? I vaguely remember that openwfd (maybe from your FOSDEM talk?) just worked with a very specific intel card.
    Is that still the case?
    Is there any USB wifi device that works?

    Reply
    1. David Herrmann Post author

      ath9k-htc based dongles work fine. I use the UWA-BR100, but it is quite expensive. However, given that many manufacturers often change chipsets on new device revisions, you can never be sure to actually order the correct device.. If you know other available devices with ath9k-htc, please let me know.
      Btw., ath9k devices (PCIe) should work, too.

      Regarding intel, you need iwlwifi-mvm, which is currently just the iwl-7260 mini-PCIe card. But wpa_supplicant-git is broken with that card.. some hacks are needed to make it work.

      Reply
  5. liberforce

    Hi David, it would be nice if you could put your FOSDEM slides somewhere, as the sound on that presentation in quite bad and makes it hard to follow. Thanks for the good work !

    Reply
    1. David Herrmann Post author

      I’m working with Tom on networkd (well, I’m just pointing out bugs, but lets call that “work”). But networkd is network-layer and up, anything below it should be handled externally. That means, a wifi-daemon would setup the link-layer and networkd anything on top of that. The kernel provides sufficient notifications here that there’s no need to combine both.

      For instance, for wifi you’d start wpa_supplicant (or any replacement) and once the STA is associated, networkd gets a kernel-event and can setup the network-layer.

      Reply
  6. yell

    I installed system-stable-209 and the libwfd on my fedora 20 distro.
    But trying to run “make” for miracled I recieve the following error:

    undefined reference to `sd_event_source_unref’ …. and so on

    I guess there is a missing lib.

    Loaded libs in this content are:

    libsystemd.so.0 (libc6,x86-64) => /lib64/libsystemd.so.0
    libsystemd.so (libc6,x86-64) => /lib64/libsystemd.so
    libsystemd-login.so.0 (libc6,x86-64) => /lib64/libsystemd-login.so.0
    libsystemd-login.so (libc6,x86-64) => /lib64/libsystemd-login.so
    libsystemd-journal.so.0 (libc6,x86-64) => /lib64/libsystemd-journal.so.0
    libsystemd-journal.so (libc6,x86-64) => /lib64/libsystemd-journal.so
    libsystemd-id128.so.0 (libc6,x86-64) => /lib64/libsystemd-id128.so.0
    libsystemd-id128.so (libc6,x86-64) => /lib64/libsystemd-id128.so
    libsystemd-daemon.so.0 (libc6,x86-64) => /lib64/libsystemd-daemon.so.0
    libsystemd-daemon.so (libc6,x86-64) => /lib64/libsystemd-daemon.so

    Is there anyone missing or what could be the issue?

    Reply
    1. David Herrmann Post author

      You need to pass –enable-kdbus to systemd to enable the experimental API. But note that you have to use systemd-git as the APIs change regularly. I recommend waiting until the systemd APIs are stable.

      Reply
  7. Mantas

    On the topic of API instability, it won’t build with systemd 212 because some functions were renamed/changed in commit 6a0f1f6d5af7c73:

    src/wifi/miracle_wifid-wifid-supplicant.o: In function `supplicant_start’:
    src/wifi/wifid-supplicant.c:2356: undefined reference to `sd_event_add_monotonic’
    ./.libs/libmiracle-shared.a(libmiracle_shared_la-wpas.o): In function `wpas_attach_event’:
    src/shared/wpas.c:1557: undefined reference to `sd_event_add_monotonic’

    Reply
  8. Gary Mort

    Hey David, I just ran across an article on Phoronix about miraclecast and it has excited me again for my own pet project – I’m especially excited that you mentioned “Therefore, we’re not limited to Wifi-P2P, but can use Ethernet, Bluetooth, AP-based Wifi and basically any other transport with an IP layer on top. This is especially handy for testing and development.”

    I’ve been hacking around with the Beagle Bone Black, a small embedded dev board which has an HDMI out interface. It also has an OTC USB interface and the sample USB ethernet driver works perfectly. That means that I can plug the BBB into my PC usb port and my PC will think it is a std usb ethernet card – while on the BBB side it also is emulating a usb ethernet card and running a dhcp server so the two devices get their own private network link.

    What I want to do is add in the final piece so that I can attach a monitor to the BBB and from my PC it gets treated as just another monitor[albeit connected via usb instead of hdmi or vga].

    From what you have described, if I add in some ethernet driver controls to miraclecast, then it can use that connection just as easily as wifi.

    I’ve been looking at the code from http://cgit.freedesktop.org/~dvdhrm/miracle/ and from http://cgit.freedesktop.org/~dvdhrm/libwfd/ and I’m not seeing the parts which configure it to act as a source or a sink. From your comments above about it being a ‘bit of a hack’ I’m wondering if you could point me to the code that does that part so I can hack around with it…. ie I can grab a wifi usb card for my BBB and try to get the wifi part working, then transition to usb.

    Reply
    1. David Herrmann Post author

      Sounds like a nice pet-project. But as you noticed, until now I only pushed the wifi-related code. All the RTSP/streaming layers are only on my machine. I am working on a thesis right now and will push the code once it’s properly designed. I don’t have as much time as I hoped, so please bear with me.

      Reply
  9. stic

    Hello David,
    thanks for your fantastic contribution, all of this seems so great.
    Is it already possible to send an already encoded stream to the sink (using a gstreamer stream as source for example) ? I didn’t found information about how to do this ?
    Thank you

    Reply
    1. David Herrmann Post author

      It’s all still under development, I’m sorry. There’s no public release, yet. If you know ‘C’, you can dive into the code and play around with it. But it’ll take some more time (and developer effort) to get a first release done.

      Reply
      1. jerry fu

        hi David,
        I am trying to compile the miracle project, but get some errors, is there anything wrong ?
        The version of systemd is 217, the latest one, since 208 or 209 does not work.

        src/wifi/miracle_wifid-wifid-supplicant.o: In function `supplicant_group_free’:
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:182: undefined reference to `sd_event_source_unref’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:197: undefined reference to `sd_event_source_unref’
        src/wifi/miracle_wifid-wifid-supplicant.o: In function `supplicant_spawn’:
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2366: undefined reference to `sd_event_source_unref’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2369: undefined reference to `sd_event_add_child’
        src/wifi/miracle_wifid-wifid-supplicant.o: In function `supplicant_timer_fn’:
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2434: undefined reference to `sd_event_source_set_enabled’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2416: undefined reference to `sd_event_source_set_time’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2417: undefined reference to `sd_event_source_set_enabled’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2428: undefined reference to `sd_event_source_set_time’
        /home/jerry/miracle/src/wifi/wifid-supplicant.c:2429: undefined reference to `sd_event_source_set_enabled’

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s