Jan Grulich

How to enable and use screen sharing on Wayland

Two days ago I wrote about our work on screen sharing in web browsers. While there was a lot of work done recently on this area, it’s not still in the state where everything would just work out of the box. There are few steps necessary to make this work for you and here is a brief summary what you need. This is not a distro specific how to, but given I use Fedora 28 and I know that everything you need is there, it’s most likely you will need to figure out the differences for your distribution or build it yourself.

PipeWire

PipeWire is the core technology used behind all of this. In Fedora you just need to install it, it’s available for Fedora 27 and newer. Once PipeWire is installed, you can just start it using “pipewire” command. If you want to see what’s going on, you can use “PIPEWIRE_DEBUG=4 pipewire”  to start PipeWire with debug information. For Fedora 29, there is a feature planned for PipeWire which should make it to start automatically.

Xdg-desktop-portal and xdg-desktop-portal-[kde,gtk]

We use xdg-desktop-portal (+ backend implementation) for communication between the app requesting to share a screen and between desktop (Plasma or Gnome). You need xdg-desktop-portal, which is the middle man between the app and backend implementation, compiled with screencast portal. This portal will be build automatically when PipeWire is present during the build. In Fedora you should be already covered when you install it. For backend implementation, if you are using Plasma, you need xdg-desktop-portal-kde from Plasma 5.13.x, again compiled with screencast portal, which is build when PipeWire is present. For Fedora 28+, you can use this COPR repository and you are ready to go. I highly recommend using Plasma 5.13.2, where I have some minor fixes and if you have a chance, try to compile upcoming 5.13.3 version from git (Plasma/5.13 branch), as I rewrote how we connect to PipeWire. Previously our portal implementation worked only when PipeWire was started first, now it shouldn’t matter. If you use Gnome, you can just install xdg-desktop-portal-gtk from Fedora repository or build it yourself. You again need to build screencast portal.

Enabling screen sharing in your desktop

Both Plasma and Gnome need some adjustments to enable screen sharing, as in both cases it’s an experimental feature. For Gnome you can follow this guide, just enable screen-cast feature using gsettings. For Plasma, you need to get KWin from Plasma 5.13.x, which is available for Fedora in the COPR repository mentioned above. Then you need to set and export KWIN_REMOTE=1 env variable before KWin starts. There is also one more thing needed for Gnome at this moment, you need to backport this patch to Mutter, otherwise it won’t be able to match PipeWire stream configuration with the app using different framerate, e.g. when using Firefox.

Edit: It seems that exporting KWIN_REMOTE=1 is not necessary, it probably was only during the time when this feature was not merged yet. Now it should work without it. You still need KWin from Plasma 5.13.

Start with screen sharing

Now you should be all set and ready to share a screen on Gnome/Plasma Wayland session. You can now try Firefox for Fedora 28 or Rawhide from this COPR repository. For Firefox there is a WebRTC test page, where you can test this screen share functionality. Another option is to use my  test application for Flatpak portals or use gnome-remote-desktop app.

Edit: I didn’t realize that not everyone knows about xdg-desktop-portal or PipeWire, below are some links where you can get an idea what is everything about. I should also mention that while xdg-desktop-portals is primarily designed for flatpak, its usage has been expanded over time as it perfectly makes sense to use it for e.g. Wayland, where like in sandbox, where apps don’t have access to your system, on Wayland apps don’t know about other apps or windows and communication can by done only through compositor.

 

Screen sharing in Plasma wayland session

One of the important missing features in Plasma wayland session is without a doubt possibility to share your screen or record you screen. To support this you need help of the compositor and somehow deliver all needed information to the client (application), in ideal way something what can be used by all DEs, such as Gnome. Luckily, this has been one of the primary goals of Pipewire, together with support for Flatpak. If you haven’t heard about Pipewire, it’s a new project that wants to improve audio and video handling in Linux, supporting all the usecases handled by PulseAudio and providing same level of handling for video input and output. With Pipewire supporting this, there was recently a new API added to xdg-desktop-portal for screen cast support and also for remote desktop. Using this API, applications can now have access to your screen content on Wayland sessions or in case they are running in sandbox. With various backend implementation, like xdg-desktop-portal-kde or xdg-desktop-portal-gtk, they just need to support one API to target all desktops. Screen cast portal works the way, that the client first needs to create a session between him and xdp (xdg-desktop-portal) backend implementation, user then gets a dialog with a screen he would like to share and starts screen sharing. Once he does that, xdp backend implementation creates a Pipewire stream, sends back response to the client with stream id and then client can connect to that stream and get its content. Once he no longer requests content of the selected stream, xdp backend implementation gets information that nobody is longer connected to the created Pipewire stream and can stop sharing screen information and xdp backend implementation is again ready to accept next requests for screen sharing. This is all happening in the background so there is really no cool picture I can show, at least this dialog which you get when you request to share a screen.

I finished support for screen cast portal in xdg-desktop-portal-kde last week and currently waiting for it to pass review and be merged to master. This is also currently blocked by two not merged reviews, one adding support for sending GBM buffers from KWin and one with new Remote Access Manager interface in KWayland, both authored by Oleg Chernovskiy, for which I’m really greatful. This all will hopefully land soon enough for Plasma 5.13. Testing this is currently a bit complicated as you need everything compiled yourself and besides my testing application there is really no app using this, except maybe Gnome remote desktop, but there should be support in future for this in Krfb, Chrome or in Firefox. Hopefully soon enough.

Last thing I would like to mention is for GSoC students. We also need remote desktop portal support to have full remote desktop experience so I decided to propose this as a GSoC idea so students can choose this interesting stuff as their GSoC work.

Gnome integration for Qt based applications in Flatpak

Following blog post from Patrick Griffis about new themes support in Flatpak, we started working on supporting this new feature too. Currently wherever you start a Qt application, it would always look like a KDE application or something would be missing, like icons so you would end up with bad experience and mixed feelings. This is going to change now as we now support Gnome in form of icons, widget style and Qt platform theme and with this, when you run a Qt application in Gnome, it will look definitely better and more natively than before. We packaged regular adwaita icons which are used by default in Gnome as extension of freedesktop runtime. For widget style we use adwaita-qt style, which is a Qt style attempting to look like Gtk’s adwaita and the most important part putting this all together is QGnomePlatform, a Qt platform theme which reads your Gnome configuration and applies it to running Qt applications. QGnomePlatform also enforces Qt apps to use adwaita icons and adwaita-qt style by default so that’s another reason why it is important. Both adwaita-qt and QGnomePlatform projects are by the way authored by Martin Bříza, a collegue of mine from Red Hat so if you meet him in person somewhere buy him a beer for that he cares about Qt integration in Gnome :). Now coming to a question how to install this and make it work. Basically all you need to do is install following extensions and you shold be done:

flatpak install kderuntime org.freedesktop.Platform.Icontheme.Adwaita
flatpak install kderuntime org.kde.KStyle.Adwaita
flatpak install kderuntime org.kde.PlatformTheme.QGnomePlatform

Your Qt apps running in flatpak should then automatically pick up all of these extensions without any further modification, same way it does automatically when you run it outside the sandbox. Simply done!. I’m also aware that there are more Gtk themes, like adwaita-dark or high-contrast, both are also available in form of Qt style and we will probably package them later, but at this point it is mostly proof of concept that this can be done and works nicely. You can follow our wiki page if you want more information about runtimes, repository with applications and so on and from me it’s all for now.

Btw. below you can see okular running in flatpak and using adwaita-qt style with adwaita icons.

Telegram desktop client for flatpak #3

I have been working lately on fixing issues we had with telegram when using Qt’s flatpak platform plugin to have portals support. This was all crashing due to system tray. Qt tries to fallback to xembed when platform theme doesn’t provide any system tray implementation. Unfortunately Qt assumes that we are using xcb platform plugin (used everywhere by default) and tried to call some functions on it, but unfortunately we were using flatpak platform plugin which loads xcb only internally and because of that it was crashing. To solve that I had to implement my own xembed support into our flatpak platform plugin without using xcb specific functions. This was all happening only on Gnome, because in KDE our platform theme has it’s own implementation of system tray using SNI and xembed is not used there at all. With all those fixes I decided to make telegram to finally use flatpak platform plugin by default.

What is also new is that I created one more branch with alpha releases. Right now I have master branch with latest stable telegram in version 1.0.29 and devel branch with latest alpha release 1.0.37 bringing support for calling.

You can get it the same way as before, following instructions from previous blog post, except that now you have to specify branch you want to install.

Discover: Open desktop ratings

As I’m trying to keep improving the flatpak backend in discover I decided to add support for reviews. To do so I implemented support for GNOME’s Open Desktop Ratings which is rating/review system used by gnome-software. Result of this is now fully functional review system, where you can read user comments and ratings and submit your own reviews. We also use same mechanism as in gnome-software for generating user_hash which identifies you in odrs server and given that you are able to modify/delete your reviews from both discover and gnome-software (note that discover doesn’t support this yet). You can also vote for already existing reviews so others get feedback on how useful each review is. We also decided to use same review system in our PackageKit backend and replace current Ubuntu Popularity Contest system so not only flatpak users will benefit from this. During testing of this review support we’ve hit many UI issues related to review system causing users not to be able see reviews or write new ones which were introduced during transition to kirigami. We fixed all of them and you can look forward to improved experience in the upcoming discover release. To improve this further, we or at least I, would like to also add a new widget showing current total app ratings as of now you can only see reviews with comments only, not overall app rating, but this needs some discussion and design consideration.

Here are some screenshots to prove that this is already working:

If you want to test it, which we would like you to do, you can just compile discover from master branch (with -DBUILD_FlatpakBackend=ON cmake parameter for flatpak support). That’s all from me for now. Have a nice weekend :).

KDE Flatpak portals introduction

I guess you all have heard about Flatpak, Snappy and sandboxing in general. Flatpak is a new way of distributing applications. With Flatpak applications are running in sandbox, which means they are isolated from the rest of your system. With that in mind you need a way how to access some stuff outside the sandbox, like your files or have access to your hardware. To solve this problem the Flatpak developers came up with portals. Portals are high-level session bus APIs that provide access to resources to sandboxed applications. Idea is that there is one DBus service available and visible for the sandboxed application which is supposed to communicate with it to get access outside the sandbox. Then this one service communicates with backend implementations which may be different per desktop so you have in example a Gnome implementation or in my case KDE implementation. The backend then provides dialogs so users can access files or hardware outside the sandbox. To add portal support you need to add your backend implementation, which is quite easy part and if you don’t have any then one from other available ones will be used. Complicated part is to alter the framework the application is using to use the portal DBus service instead of doing what it usually do (e.g. when opening a file dialog you want to send a request over DBus instead of actually displaying the dialog). I’ve been playing with this for some time and I tried to cover the most common or required portals. Let’s go through all of this:

File chooser portal

I think this is the most common and needed portal from all of them. I’ve added a backend implementation as well as support for this to Qt in form of my own Qt platform plugin. Given you can modify Qt’s behaviour using your own Qt platform plugin then I didn’t have to modify Qt at all. The platform plugin alters FileDialog to talk around DBus instead of showing the dialog. The dialog is then shown thanks to the backend implementation and user running app in sandbox shouldn’t notice any difference. The file chooser portal supports both opening and saving files with possibilities to set properties like filters, accept label text etc. which are all already implemented in both like in the platform plugin and the backend.

App chooser portal

This portal allows the user to select an application outside the sandbox which should be used for opening a document, a file or whatever. It is also used when opening an url. On the backend side this is just a simple dialog with list of possible applications associated with given file/document type. On Qt side I added my own implementation of OpenUrl() again into the platform plugin to make it transparent the same way the file dialog works.

Print portal

I guess the most complicated and also quite important portal. I’ve been able to add just backend implementation so far which will be used for printing from gtk applications as gtk already supports printing from sandbox. The idea behind this portal is that when app requests to print a document, it calls PreparePrint() method and the backend presents classic print dialog to the user where he can configure printer, setup the page and paper and so on. This configuration is then passed back to the application where the framework is supposed to create a pdf or ps file already pre-formatted and ready for printing. This file is then passed as file descriptor again to the backend using Print() method and printed. This all works with Qt backend and gtk app, or Gnome backend and gtk app, but unfortunately not yet with Qt apps as I still don’t know how to do this without touching Qt code as this cannot be done on platform plugin level, at least from what I can see. A simple solution can be to don’t touch Qt at all and let the app print the document to file and have a simple utility sending this file through the portal to print it. As you can set application name which should be used for printing in QPrinter, then we maybe can just set this automatically when the app is running in sandbox to make this work automatically. I’m definitely open to your ideas or any help :).

Notification portal

Again very useful portal and not that complicated. I have full backend implementation presenting a notification outside the sandbox when someone calls AddNotification() method. To make this work for KDE applications automatically I had to modify KNotifications framework and implement my own flatpak plugin which replaces NotifyByPopup plugin. All this plugin does instead displaying a popup it calls the portal DBus service which then calls AddNotification() in my backend and presents notification outside the sandbox. Both the plugin and backend supports also sending back information about triggered action so you can also get feedback back to the sandboxed application.

Those are all the portals I have fully or partially covered so far. There are of course more portals designed in Flatpak portal API, like screenshot portal, inhibit portal, but for some of them we don’t have any framework/API or they are not that important so they will be added later. I also have a test application which you can run in sandbox and test all the portals.

How to use flatpak portals

In order to use flatpak portals, either my implementation or the Gnome one, you need to install xdg-desktop-portal which is the main portal DBus service visible to sandboxed applications and which calls backend implementation. For gtk implementation you need xdg-desktop-portal-gtk and for KDE/Qt implementation you need xdg-desktop-portal-kde. This is required to be installed outside the sandbox to present dialogs for file and hardware access. To add support for portals to your sandboxed application you should be fine already with gtk, with Qt and KDE you need my Qt platform plugin and my modifications made to KNotifications. To use those you need to modify flatpak manifest to include them during build so they are available in the sandbox. You can get inspiration in my test app manifest. And finally to use my platform plugin as KNotifications will be used automatically you need to start your app using flatpak run your_app -platform flatpak.

I guess that’s all from me today. Patches and improvements are warmly welcomed as well as any reported issue. If you want some information about Flatpak and KDE in general, we have setup a wiki page where you can find information about KDE runtimes and applications packaged for Flatpak. There will be also a talk at FOSDEM this year about Flatpak and KDE from Aleix Pol.

Scroll To Top