Last time I wrote about possibility to share a screen of Plasma wayland session, using xdg-desktop-portal and our xdg-desktop-portal-kde backend implementation. Problem was that during that time, there was no application which would implement support for this, leaving my previous effort useless so far. Luckily, this should change pretty soon. I, together with my Red Hat collegues Tomáš Popela and Eike Rathke, have been working for past few weeks on bringing support for screen sharing on Wayland to web browsers. All modern browsers use WebRTC for all audio-video communication, including screen sharing, meaning that in a perfect world, just one implementation would be needed, which is not that exactly this case. Let’s go a bit into the details first.
Each system (Windows, Mac and X11) in WebRTC reimplements an abstract class called DesktopCapturer, which defines API used by applications to support screen sharing. For our wayland support, we started with a new implementation using Pipewire as the core technology used for screen content delivery and for the communication part, to request which screen to share and to obtain Pipewire stream information, we use xdg-desktop-portal, providing simple API to do so. Advantage of using xdg-desktop-portal is that it will work also in sandbox (Flatpak and Snap) and that there is support in Plasma (using xdg-desktop-portal-kde backend) and support for Gnome (using xdg-desktop-portal-gtk backend), both using same API. Using our desktop capturer implementation, WebRTC starts to communicate with xdg-desktop-portal, we set up a session associated with our request, we tell xdg-desktop-portal that we have interest in screen sharing so xdg-desktop-portals asks your backend implementation to provide a dialog to select a screen to share and once this is settled, we request to start screen sharing and backend implementation will set up Pipewire stream, sending us back file descriptor of a Pipewire remote which we can open and connect to it. Once we are connected, we finally start receiving buffers from Pipewire with screen data and providing them to applications. This so far sounds simple and that the work is basically done, but this is unfortunately not an ideal world.
We found out that e.g. Firefox uses some older copy of WebRTC, so while working on WebRTC trunk, to have our work ready for upstream, we had to modify slightly our changes for Firefox older copy in order to be able to test our changes. There is also one thing we haven’t figured out yet, the thing is that Firefox has its own dialog to select a screen (for other capturer implementations) and we were not able to avoid displaying it when our capturer is used. Another thing is Chromium, which seems to be using WebRTC in a different way when compared to Firefox as Chromium is using plugins so this is also something we have to figure out. There are probably still plenty of other things to do before all of this can be upstream, but we have made great progress on this so far. We even had couple of Bluejeans calls where both sides could share their screens, one running Gnome Wayland session and me running Plasma Wayland session.
And for those who like adventures, I have set up a Fedora COPR repository (currently building), with Firefox containing our changes so you can test it yourself, we will need testers soon or later anyway. You just need to make sure that Pipewire is running, otherwise this won’t work, portals (both xdg-desktop-portal and backend implementation) should be started automatically.
Screenshot of Firefox in action:
7 thoughts on “Screen sharing support in WebRTC for Wayland sessions”
Nice work 🙂 Will this only be used for screensharing as in “video” or will this also be used as a replacement for a VNC session so you can interact with the desktop?
Yes, there is another portal in xdg-desktop-portal for remote desktop (only implemented in Gnome backend so far) which is intended for interaction. This one, combined together with the one for screen sharing is already used by e.g. gnome-remote-desktop.
Would this allow sharing only one application? (That is what the screenshot seems to show).
The screenshot shows my full screen, including panel. Currently only full screen content can be shared, support for single applications/windows needs to be implemented to both portal backends and to KWin and Mutter.
Impressive. Is there any instruction how to test it?
At the moment I compiled your test app, but when I click on the “request” button and select a monitor, nothing happens. The app logs:
Failed to start: 2
I didn’t realize there are other steps necessary in order to test this. I’m now reworking how xdg-desktop-portal-kde initialize Pipewire so it’s more reliable and once I finish it, I’ll write a post about what’s needed and how to use it.
Awesome, thank you. At this stage I have to idea how to start “remote sharing” and how to connect a client (your build of FF) in order to test it.