While we have pretty good support for screen sharing on Wayland in WebRTC, which is included in browsers like Chromium or Firefox, it is still not enabled by default in Chromium and it is kept behind a flag. Not only you have to remember to always enable it for new configurations, but for many users it is not even something they are aware of. This has been my main focus recently and I would like to share with you steps that has been done and what are the plans for the future.
What are the changes to expect in Chromium soon?
DMA-BUF improvements/fixes:
Last year I landed proper DMA-BUF support in WebRTC, which made things way faster. It was working, but it was not perfect and there were some corner cases where it might not be working at all. Here are changes I made recently:
- Advertise DMA-BUF support when it is really supported. Older versions of PipeWire don’t handle the new way of DMA-BUF negotiation and therefore it shouldn’t be used in such cases. Also using DMA-BUF modifiers requires some recent versions of PipeWire on both sides.
- Implemented stream renegotiation. In situation when we fail to import a DMA-BUF with given modifier, we will drop this modifier and try to renegotiate stream parameters and go with a different modifier or fallback to shared memory buffers in case we fail completely.
- Make sure to import DMA-BUF with correct render node. In case of multi-gpu setups, we always picked the first render node to import DMA-BUFs, but it can happen that they were actually produced by a different render node and for that reason we might fail to import them. We now try to get default EGLDisplay, which should be the same one used by the wayland compositor and we should be using same render node.
Better mouse cursor support:
Until now we had mouse cursor as part of the screen content. This means that everytime you moved with your mouse cursor, we had to update whole image and that is very inefficient. The API in WebRTC allows you to implement MouseCursorMonitor which can be used to track mouse changes only and each platform can have both MouseCursorMonitor and DesktopCapturer implementations combined in DesktopAndCursorComposer to get complete image and this all works automatically like a magic. Unlike X11 implementation, our only option is to get everything from one PipeWire stream we connect to and there was no way how to make it shared from DesktopCapturer implementation so it can be used by MouseCursorMonitor implementation. I had to split DesktopCapturer to have xdg-desktop-portal and PipeWire separate implementations. Code for PipeWire is now a SharedScreenCastStream class which is being shared through DesktopCaptureOptions. This is set of parameters associated with each capturer instance and luckily this is also passed to MouseCursorMonitor so we can have access to already initialized PipeWire stream and get the cursor data from there. Implement MouseCursorMonitor with SharedScreenCastStream was then piece of cake.
List of merge requests:
- https://webrtc-review.googlesource.com/c/src/+/238174 (split xdg-desktop-portal and PipeWire implementations)
- https://webrtc-review.googlesource.com/c/src/+/239362 (use ScreenCaptureFrameQueue)
- https://webrtc-review.googlesource.com/c/src/+/244507 (implement MouseCursorMonitor)
This should again significantly improve performance of screen sharing, because moving with a mouse over a static screen content doesn’t need full screen content update.
Misc:
Last but not least, I’m now in touch with Google developers who help me to review all my changes and discuss with me the current state, issues I have, etc. on monthly meetings we have. The plan is to make this finally enabled by default, hopefully in the first half of this year. There are still some things that need to be solved before this is enabled and there is lot of work ahead, but things look promising.
Plans for the future:
- Implement stream restoration
- this will allow us to skip the second portal dialog and I already have plan in my head how to do this in WebRTC. This is currently only supported by xdg-desktop-portal-gnome and xdg-desktop-portal-kde lacks this functionality.
- Improve UX of the Chromium screen sharing dialog
- Write tests for all PipeWire/portal code in WebRTC
Even though WebRTC is used in Firefox, I mostly talk about Chromium, because Firefox doesn’t use most recent WebRTC and will need to pick all the changes I did or rebase to newer WebRTC in order to have them. Firefox also has PipeWire/Wayland screen sharing enabled by default and doesn’t have UX issues as there is no internal screen sharing dialog like in Chromium.
I hope all these changes will make your experience better and next time when you read a new blog post I will be informing you about end of this journey.
Exciting update. Thanks for your hard work on this! Particularly making it enabled by default & supporting stream restoration will really plug a real user experience gap for all the Linux desktop distros where users want to screenshare in conf calls.
Yes thanks very much Jan for undertaking this important and hard work (which I’m sure will benefit many people who are, or want to use GNU/Linux as the platform of their choice). I think there is no understatement that this kind of work really makes a huge difference in making GNU/Linux a more attractive platform 🙂
So please keep your good efforts and work up 🙂
Thank you Jan!
Very happy to hear on all the progress, as someone who uses Chrome with webrtc screensharing all the time thank you very much for all your work on this, it is *very* much appreciated!
Thank you for your awesome work!
> Even though WebRTC is used in Firefox, I mostly talk about Chromium, because Firefox doesn’t use most recent WebRTC
Could you please elaborate a little bit which features of the “recent WebRTC” are missing in Firefox? Is there also a list of those to track (like in Mozilla’s Bugzilla)?
> will need to pick all the changes I did or rebase to newer WebRTC
Do you see that coming? Are there plans from Mozilla to do so?
All the changes from my last two WebRTC blog posts. That means DMA-BUF support and the new separate mouse cursor implementation. Firefox has more than a year old copy of WebRTC. There was a bug for WebRTC rebase, but I think there was no one actively working on it. At least Fedora users have Firefox on par with Chromium as I updated the package there.
Great work, and great talk at LAS 2022!
Thanks for the great work! As a Fedora user using Microsoft Teams (web) for work I definitely benefit from these improvements. Really looking forward to stream restoration and better UX 🙂
Hey Jan,
Thanks for making screen sharing work in Chromium with multi-gpu setup! I switched back to Wayland from X11. Even tho Slack haven’t updated electron to the latest version yet (to take advantage of your changes), at least I can screen share from Chromium.
TBH, I would’ve been happy to use Firefox for that (in fact I use Firefox for everything but meetings), but unfortunately background blurring in Google Meet (and Zoom) doesn’t work in Firefox.
Hi Jan, thanks for your great work. One questions please, why does XWayland still run as a process when I am screensharing via Jitsi or Teams? With “WebRTC PipeWire support” enabled, “WebRTC will use the PipeWire multimedia server for capturing the desktop content on the Wayland display server.” XWayland shouldn’t be necessary to run the PipeWire portal. This issue doesn’t happen with flatpak versions of Chrome or Firefox, just with rpm versions.
Hi, my guess is that the official Chrome still runs as an XWayland app, not natively on Wayland. Firefox runs natively on Wayland by default and I guess that Chrome/Chromium from Flathub also defaults to Wayland by default.
Hi Jan, thanks for the response. Just to clarify, this issue (XWayland running when screensharing) happens when running either Chrome or Firefox natively as Wayland clients. Only the rpm versions are impacted. Try for yourself and see.