Jan Grulich

WebRTC/Chromium updates in 2020

In 2019, I started with my first contribution to WebRTC. This was all about screen sharing support on Linux Wayland sessions, using xdg-desktop-portal and PipeWire. Back then, it was quite simple, we only had PipeWire 0.2 and all portal backends supported only screen sharing (no window sharing). While this was relatively easy, it was not ideal as each screen sharing request involved two portal dialogs to get the screen content on the web page itself. For me it was a big success, because I made quite a significant contribution to such a big project, which is used by many people, and a project which is used by all modern web browsers.

At the beginning of 2020, the year everyone would like to erase from their memories, we got PipeWire 0.3 (with slightly different API) and later with xdg-desktop-portal-gtk and xdg-desktop-portal-kde (later this year) people were finally able to share application windows. Support for all of this was lacking in WebRTC, because back then those were not available. I wanted to tackle all issues at once, bring support for window sharing and get rid of the “dialog hell” with portals, which was even worse with the new window sharing capabilities in portal backends.

This is what the situation looks like. With each request to share a screen, you got the preview dialog from Chromium. This dialog consists from three pages. One is for screen sharing making one portal request, second one is for window sharing, which is another portal request, and the last one is just to allow you to share a web page you have opened. You had to confirm both portal dialogs, then confirm the Chromium dialog and finally you got one more portal dialog (ouch) to get the screen content on the web page itself.

I had a solution. I made all portal calls identified with an ID and shared this ID (portal call) in Chromium between both pages in the Chromium preview dialog and with the request made for the web page itself. With this solution we only had ONE portal dialog. This was a perfect solution (at least seemed to be). I started working on this at the beginning of this year, we exchanged many emails with people from Chromium UX team, because I wanted to do also some minor UI changes in the preview dialog. Unfortunately, those were rejected for consistency with all platforms. It was not a big deal and I submitted my changes for review, keeping UI as it was, just adding all necessary bits into Chromium and WebRTC to make it all work.

I wish to say things went smoothly since then, but the opposite is true. It took a while to get everything reviewed, but this is probably no surprise with this year being weird and many people working from home with less than ideal conditions. Anyway, few months passed away, I ended up rewriting my changes many times, not even counting hours I spent on it. This all resulted into me being obsessed with this change, it mattered to me so much to get it merged. I was constantly thinking about how to make it better, I was many times fixing issues in the evening (as reviewers were mostly US based), instead spending time with my family. It would be even better to waste my time with my beloved Playstation. This had really negative impact on my mental health and I realized this has to stop and I simply gave up, because I couldn’t continue this way and needed a break. I abandoned both changes (WebRTC and Chromium) and decided to just pick changes I will be able to successfuly upstream. I probably made my change too ambitious and complicated or maybe it’s just Chromium not being ready for this kind of change, because some tweaks were specific for my use-case. It’s also hard to say I wish upstream devs had helped me more, because there is so much to understand around Wayland, portals and PipeWire and way how it all works together.

Anyway, with a new start, without pressure after gaving up on the change, I picked the most important changes and submitted them separately. I was surprised now how smoothly this went and how fast those changes were upstreamed. Simply those changes were simple, understandable and easy to review. I didn’t gave up on fixing the “dialog hell” completely, I have some other ideas, but next time I will try to submit them step by step and will keep some distance and my free time.

And what are the changes you can expect in upcoming Chromium release in 2021?

Support for PipeWire 0.3

You can now build Chromium/WebRTC with both PipeWire 0.2 and Pipewire 0.3. There is a new “rtc_pipewire_version” option you can pass to your build configs.

Window sharing support

There is probably no description needed. You will be able to share application windows in case you don’t want to share whole screen.

Suppport for DmaBuf and MemFd buffer types

This should allow faster transfer of your screen content from your Wayland compositor, through PipeWire to your browser.

Less portal dialogs involved

If you look back into the screenshot I posted above, you can see there are two portal dialogs opened just for the Chromium preview dialog. I at least tried to reduce this to just one portal dialog. This was done by removing the page for window sharing, because the screen share request will already handle both screen and windows.

I think you can expect above mentioned changes in Chromium 89 and I hope you will at least appreciate some of these improvements even though I didn’t deliver everything I wanted to. Also, thanks to Martin Stránský from our Firefox team, you can expect all these changes to be also part of Firefox.

Happy holidays and see you in a better year.

Refreshed UI for Fedora Media Writer

For those who don’t know, Fedora Media Writer is a tool to create bootable live USB drive with your favorite flavor of Fedora. It is written in C++ with UI written in QML and it is supported on Linux, Windows and Mac OS X. It was developed by Martin Bříza, my former collegue from Red Hat, who did an amazing job in the past. Fedora Media Writer (FMW) primarily targets Fedora Workstation and therefore the UI looks like a GNOME app using Adwaita theme. Unfortunately the Adwaita theme changed over time and originally FMW was written using QtQuickControls 1 (deprecated these days) so it needed an UI overhaul.

I started working on FMW during the summer, slowly migrating it to QtQuickControls 2. The original UI had lots of custom QML widgets, basically standard widgets with Adwaita skin on it. I still wanted FMW to use Adwaita theme, because Qt doesn’t have any native QML components for Windows, Mac OSX or GNOME and writing those would require lots of work. Therefore I decided to write a new QQC2 based Adwaita theme which can be used on all platforms. To avoid duplicating half of the code we already have in Adwaita-qt (a QStyle to make QWidgets look like Adwaita), like information about widget sizes and colors, I reworked Adwaita-qt to provide a library so it can be used by projects like this and so they don’t need to update everytime Adwaita changes. It was more work than I anticipated because it needed quite a lot of changes to separate things into library and also to make it build and work on all platforms where I want to use FMW. Good news is that the work is now done and I made a pre-release of Adwaita-qt. The library for now provides information about widget sizes, color palette and colors used by all widgets, but I plan to extend this in future with addition of Adwaita-qt rendering part allowing the library to render basic widgets for you. That’s something I would like to use for example in QGnomePlatform (GNOME platform theme) to render buttons in window decorations. With a lot of information being already said about Adwaita-qt, the work on QQC2 Adwaita theme was an interesting experience and probably the most enjoyable one, because everytime you write a new component and port the app to use it, you see the result of your work and the app slowly migrating towards a more modern UI makes you happy with the result. I don’t know what more to say about the QQC2 Adwaita theme as it’s basically QML variant of widgets we have in Adwaita-qt, with difference that it should look exactly the same on all platforms thanks to using Adwaita-qt. In past with QQC1 all the colors were derived from system QPalette making it slightly different on all platforms. If you wonder why the QQC2 theme is not part of Adwaita-qt, where it will most likely end up, then it’s because it’s not complete yet and contains only components used in FMW itself. Anyway, I have finished the port to QQC2 this week with some late fixes and after I spent a week updating all build systems (Windows, Github CI, Mac OSX) to properly build and produce builds for you to test since I made a new pre-release yesterday \o/.

The work on this port is most likely not 100% finished as I expect some minor issues to appear here and there, but I tried to make this 1:1 copy of the previous version so don’t expect any major changes. I will be glad if you try it and let me know what you think. Thank you and especially big thanks goes to Martin Bříza for his help during the development and for the work he did on this project in the past.

You can get it from following locations:

Here you have some images for comparison:

Cute Qt applications in Fedora Workstation

Fedora Workstation is all about Gnome and it has been since the beginning, but that doesn’t mean we don’t care about Qt applications, the opposite is true. Many users use Qt applications, even on Gnome, mainly because many KDE/Qt applications don’t have adequate replacement written in Gtk or they are just used to them and don’t really have reason to switch to another one.

For Qt integration, there is some sort of Gnome support in Qt itself, which includes a platform theme reading Gnome configuration, like fonts and icons. This platform theme also provides native file dialogs, but don’t expect native look of Qt applications. There used to be a gtk2 style, which used gtk calls directly to render natively looking Qt widgets, but it was moved from qtbase to qt5-styleplugins, because it cannot be used today in combination with gtk3.

For reasons mentioned above, we have been working on a Qt style to make Qt applications look natively in Gnome. This style is named adwaita-qt and from the name you can guess that it makes Qt applications look like Gtk applications with Adwaita style. Adwaita-qt is actually not a new project, it’s been there for years and it was developed by Martin Bříza. Unfortunately, Martin left Red Hat long time ago and since then a new version of Gnome’s Adwaita was released, completely changing colors and made the Adwaita theme look more modern. Being the one who takes care of these things nowadays, I started slowly updating adwaita-qt to make it look like the current Gnome Adwaita theme and voilà, a new version was released after 3 months of intermittent work. You can see the results here:

Isn’t it beatiful? The theme is far from being perfect and there will definitely be still some minor issues, but writting a Qt style is far from being an easy task as the QStyle class is quite complex. If you find any issue, you can open a bug and I will try to fix it. You can also send me patches if you decide to fix something yourself (I will be happy for that). The repository is hosted on GitHub under FedoraQt/adwaita-qt.

And of course it was a lie, the screenshots above are the old version of adwaita-qt (for comparison), the new ones are actually here:

I hope you like it more now :).

Tutorial: Screen Sharing and Remote Desktop on Fedora Workstation 30 (Wayland)

I recently got an email from a user asking me how to make all this work on Fedora. Problem is that unlike in old XServer sessions, there are certain things which need to be enabled first. There are also dependencies which need to be installed and services which need to be running. While most of the dependencies are automatically installed and services automatically activated, there still might be situations when this is not true, for example when switching from another desktop so it’s better to cover it all. This tutorial targets Fedora, but it can be probably used by any other distribution.

Dependencies

Both screen sharing and remote desktop work almost identically on Wayland, they both use portals as a communication tool between applications and compositor (in this case Mutter) to start the process of sharing and setup PipeWire stream (see below). While portals were primarily meant to be used by sandboxed applications (e.g. Flatpak) to get access to system (like files or printing) outside the sandbox, their design perfectly fits for Wayland usage too. In Fedora you should have portals automatically installed, they are represented by two separate packages, first is xdg-desktop-portal, which is the portal service communicating with sandboxed applications and with a backend implementation of portals, and the second package is the backend implementation, in our case xdg-desktop-portal-gtk. Both are DBus activatable, which means they are automatically started whenever application calls them. The reason why portals consist from two services is that there can be multiple backends, each one providing native dialogs for your desktop. For example you don’t get a gtk dialog to open a file in KDE Plasma session or you want a backend communicating with specific compositor (like in our case with Mutter).

The second important dependency is PipeWire. PipeWire is the core technology used for screen content delivery from the compositor to applications. This is done throught a PipeWire stream shared between the compositor and application. PipeWire should be automatically installed on your system, the package name is pipewire and it provides socket-based activation so you shouldn’t need to worry if it’s running or not.

Enabling screen sharing and remote desktop in Gnome

You don’t seem to do any additional step in order to make screen sharing work. However, you need to enable remote desktop (if you want to). Go and open gnome-control-center (Settings) and there go to Sharing section. There you should see this window when you click on Screen Sharing:

If you don’t have such option, make sure you have installed gnome-remote-desktop, because I’m not sure whether it’s installed by default. Allowing screen sharing will start a server instance which you can connect to from another computer, using vnc://linux.local or your_computer_ip:5900. You will most likely need to open a hole to your firewall, but the same you need to do for any other VNC server. I tried to connect with Vinagre and Krdc (KDE VNC viewer) and both worked for me, but I was unable to connect with Tigervnc (vncviewer) due to not maching security type.

Screen sharing in Firefox

Firefox in Fedora already comes with PipeWire support enabled and you don’t need to do anything special. You can test it for example with this testing page. The PipeWire support in Firefox unfortunately needs to be enabled during build by handmade changes, which is most likely not happening in other distributions, but from what I now there is an ongoing effort to make this configurable with a build option.

Screen sharing in Chromium/Chrome

Similar situation is with Chromium, where PipeWire needs to be also enabled during build, but it’s already configurable via a build option. In Fedora we have this enabled by default. Official Chrome builds are build with PipeWire support enabled as well. I should maybe mention that PipeWire support is in Chromium starting with version 74.

Unlike with Firefox, this support needs to be also enabled in runtime. You can do that with following chrome flag:

chrome://flags/#enable-webrtc-pipewire-capturer

Then you should be all set to be able to share a screen or a window from your Chromium or Chrome.

Issues

Don’t get scared by higher number of dialogs for screen/windows selection you will get when sharing screen in your browser. We are aware of this annoying usability issue and hopefully we will manage to solve it one day. The reason why this happens is that every browser provide their dialog for screen/window selection and in both browsers these dialogs show previews of your selection. You need to select screen/window first in the portal dialog for the preview dialog and once you accept the preview dialog in your browser, you again need to select screen/window in the portal dialog to get the content into the web page itself.

Support in other applications

There is a KDE application called Krfb, which in the next release (19.08) will have similar support for remote desktop on Wayland as you have in Gnome. Otherwise there are probably no other applications which would allow you to share a screen or control your desktop remotely on Gnome Wayland sessions. You will not be able to use TeamViewer, Tigervnc or any other application you are used to use. If you want to use these applications, you will have to switch to X session for now.

How to enable and use screen sharing on Wayland

Two days ago I wrote about our work on screen sharing in web browsers. While there was a lot of work done recently on this area, it’s not still in the state where everything would just work out of the box. There are few steps necessary to make this work for you and here is a brief summary what you need. This is not a distro specific how to, but given I use Fedora 28 and I know that everything you need is there, it’s most likely you will need to figure out the differences for your distribution or build it yourself.

PipeWire

PipeWire is the core technology used behind all of this. In Fedora you just need to install it, it’s available for Fedora 27 and newer. Once PipeWire is installed, you can just start it using “pipewire” command. If you want to see what’s going on, you can use “PIPEWIRE_DEBUG=4 pipewire”  to start PipeWire with debug information. For Fedora 29, there is a feature planned for PipeWire which should make it to start automatically.

Xdg-desktop-portal and xdg-desktop-portal-[kde,gtk]

We use xdg-desktop-portal (+ backend implementation) for communication between the app requesting to share a screen and between desktop (Plasma or Gnome). You need xdg-desktop-portal, which is the middle man between the app and backend implementation, compiled with screencast portal. This portal will be build automatically when PipeWire is present during the build. In Fedora you should be already covered when you install it. For backend implementation, if you are using Plasma, you need xdg-desktop-portal-kde from Plasma 5.13.x, again compiled with screencast portal, which is build when PipeWire is present. For Fedora 28+, you can use this COPR repository and you are ready to go. I highly recommend using Plasma 5.13.2, where I have some minor fixes and if you have a chance, try to compile upcoming 5.13.3 version from git (Plasma/5.13 branch), as I rewrote how we connect to PipeWire. Previously our portal implementation worked only when PipeWire was started first, now it shouldn’t matter. If you use Gnome, you can just install xdg-desktop-portal-gtk from Fedora repository or build it yourself. You again need to build screencast portal.

Enabling screen sharing in your desktop

Both Plasma and Gnome need some adjustments to enable screen sharing, as in both cases it’s an experimental feature. For Gnome you can follow this guide, just enable screen-cast feature using gsettings. For Plasma, you need to get KWin from Plasma 5.13.x, which is available for Fedora in the COPR repository mentioned above. Then you need to set and export KWIN_REMOTE=1 env variable before KWin starts. There is also one more thing needed for Gnome at this moment, you need to backport this patch to Mutter, otherwise it won’t be able to match PipeWire stream configuration with the app using different framerate, e.g. when using Firefox.

Edit: It seems that exporting KWIN_REMOTE=1 is not necessary, it probably was only during the time when this feature was not merged yet. Now it should work without it. You still need KWin from Plasma 5.13.

Start with screen sharing

Now you should be all set and ready to share a screen on Gnome/Plasma Wayland session. You can now try Firefox for Fedora 28 or Rawhide from this COPR repository. For Firefox there is a WebRTC test page, where you can test this screen share functionality. Another option is to use my  test application for Flatpak portals or use gnome-remote-desktop app.

Edit: I didn’t realize that not everyone knows about xdg-desktop-portal or PipeWire, below are some links where you can get an idea what is everything about. I should also mention that while xdg-desktop-portals is primarily designed for flatpak, its usage has been expanded over time as it perfectly makes sense to use it for e.g. Wayland, where like in sandbox, where apps don’t have access to your system, on Wayland apps don’t know about other apps or windows and communication can by done only through compositor.

 

Screen sharing support in WebRTC for Wayland sessions

Last time I wrote about possibility to share a screen of Plasma wayland session, using xdg-desktop-portal and our xdg-desktop-portal-kde backend implementation. Problem was that during that time, there was no application which would implement support for this, leaving my previous effort useless so far. Luckily, this should change pretty soon. I, together with my Red Hat collegues Tomáš Popela and Eike Rathke, have been working for past few weeks on bringing support for screen sharing on Wayland to web browsers. All modern browsers use WebRTC for all audio-video communication, including screen sharing, meaning that in a perfect world, just one implementation would be needed, which is not that exactly this case. Let’s go a bit into the details first.

Each system (Windows, Mac and X11) in WebRTC reimplements an abstract class called DesktopCapturer, which defines API used by applications to support screen sharing. For our wayland support, we started with a new implementation using Pipewire as the core technology used for screen content delivery and for the communication part, to request which screen to share and to obtain Pipewire stream information, we use xdg-desktop-portal, providing simple API to do so. Advantage of using xdg-desktop-portal is that it will work also in sandbox (Flatpak and Snap) and that there is support in Plasma (using xdg-desktop-portal-kde backend) and support for Gnome (using xdg-desktop-portal-gtk backend), both using same API. Using our desktop capturer implementation, WebRTC starts to communicate with xdg-desktop-portal, we set up a session associated with our request, we tell xdg-desktop-portal that we have interest in screen sharing so xdg-desktop-portals asks your backend implementation to provide a dialog to select a screen to share and once this is settled, we request to start screen sharing and backend implementation will set up Pipewire stream, sending us back file descriptor of a Pipewire remote which we can open and connect to it. Once we are connected, we finally start receiving buffers from Pipewire with screen data and providing them to applications. This so far sounds simple and that the work is basically done, but this is unfortunately not an ideal world.

We found out that e.g. Firefox uses some older copy of WebRTC, so while working on WebRTC trunk, to have our work ready for upstream, we had to modify slightly our changes for Firefox older copy in order to be able to test our changes. There is also one thing we haven’t figured out yet, the thing is that Firefox has its own dialog to select a screen (for other capturer implementations) and we were not able to avoid displaying it when our capturer is used. Another thing is Chromium, which seems to be using WebRTC in a different way when compared to Firefox as Chromium is using plugins so this is also something we have to figure out. There are probably still plenty of other things to do before all of this can be upstream, but we have made great progress on this so far. We even had couple of Bluejeans calls where both sides could share their screens, one running Gnome Wayland session and me running Plasma Wayland session.

And for those who like adventures, I have set up a Fedora COPR repository (currently building), with Firefox containing our changes so you can test it yourself, we will need testers soon or later anyway. You just need to make sure that Pipewire is running, otherwise this won’t work, portals (both xdg-desktop-portal and backend implementation) should be started automatically.

Screenshot of Firefox in action:

Telegram desktop client for flatpak

Since everyone is talking about flatpak or snappy these days I wanted to try it myself and package some application. I decided to start with telegram desktop client which was suggested to me as an application which would be nice to have packaged for flatpak. Once I saw the build instructions for the first time I thought it’s going to be an impossible task and it almost ended with that result, but I didn’t want to give up. It took me three working days to go through all dependencies and to make it build as the telegram qmake file is written with all paths hardcoded and you need to build it exactly in a way the author intended. I could of course package the binary provided by the author itself, but my intention was to provide a way to test the latest git version. Given that, I think I managed to try almost everything flatpak-builder allows you to do and I have to say I quite enjoyed doing that, because if you want to package an application for flatpak you just need to write a manifest (json file) and declare runtime and sdk you want to use, thankfully upstream already provides most common runtimes you need. Then you just need to define dependencies one by one, where for each of them you can specify whether it should be cloned from git, or if it should be downloaded as an archive or you can even use special type of source called “shell” where you can specify shell commands you want to run. Then flatpak-builder will automatically run configure (or another command you specify, like qmake) , make and make install on every source you define with build parameters you define. This everything is pretty automatized and re-running the build will skip stuff you have already built successfully before or you downloaded before. For most applications writing this manifest is relatively easy and in case you don’t need any specific library which is not provided by any available runtime you can do that pretty fast, even if you do it for the first time, just get some inspiration in another manifest for a similar application and that’s all you need.

To try the telegram desktop client I made the repo created by flatpak-builder available.
You just need to get Gnome runtime first:
$ wget https://sdk.gnome.org/keys/gnome-sdk.gpg
$ flatpak remote-add --gpg-import=gnome-sdk.gpg gnome https://sdk.gnome.org/repo/
$ flatpak install gnome org.gnome.Platform 3.20

And then you can install and run the telegram desktop client:
$ wget https://jgrulich.fedorapeople.org/telegram/keys/telegram.asc
$ flatpak remote-add --gpg-import=telegram.asc telegram-desktop https://jgrulich.fedorapeople.org/telegram/repo/
$ flatpak install telegram-desktop org.telegram.TelegramDesktopDevel
$ flatpak run org.telegram.TelegramDesktopDevel

You can of course build it yourself as well or if you want just check the manifest you can find it here:
https://github.com/alexlarsson/nightly-build-apps

Scroll To Top