Nezařazené

Starting with Fedora 41, we will make PipeWire the default backend for camera handling in Firefox. This is part of the effort to integrate support for Intel IPU6 cameras. You can read more about IPU6 cameras at the link proposing this feature, but the important part is that these cameras can only (as of now) be used with PipeWire.

Is PipeWire camera support mature enough?

Yes! At least I think so. I know there are people using it every day and I’ve been working hard to fix all the possible issues and close the gaps between PipeWire and V4L2 backends. There has also been a lot of progress in making libcamera’s software ISP work with PipeWire camera in WebRTC. This was mainly the work of Robert Mader and Kieran Bingham. Robert submitted a patch to libcamera and Kieran has helped me to verify and test changes on the WebRTC side, where I had to add support for all the additional video formats [1, 2].

Screenshot with libcamera’s software ISP in action using Firefox. Rumors are that Kieran is already using it regularly for softISP calls 🙂

There has also been amazing work done by Hans de Goede on this front. Besides all the IPU6 work, Hans fixed duplicate device entries in PipeWire’s libcamera implementation and pushed some fixes to V4L2 support in PipeWire to fix some races in device enumeration. This should fix issues with some cameras not being properly detected at startup, which affects the camera portal to properly advertise camera availability, as we rely on that Firefox. Speaking of duplicate device entries, I recently also fixed (removed) duplicate camera entries represented by IR cameras. These were previously shown as duplicate camera entries in the device list, but if the user accidentally selected the IR camera, it didn’t work at all. Another set of PipeWire camera fixes in Firefox are currently under review, mainly with some fixes and WebRTC backports and also implementing missing support for device change updates. I have to also mention Andreas Pehrson from Mozilla, who has been a great and active reviewer of all the changes in WebRTC upstream and especially in Firefox, which also contributes to the adoption of PipeWire camera support.

PipeWire camera support in Chromium

PipeWire camera support has been merged into Chromium for M127 and was implemented by Michael Olbrich from Pengutronix. To use it, you just need to go to chrome://flags and enable the PipeWire camera support flag. Unfortunately, this version is broken, because Michael and I haven’t tested it with an official build. The official build enables Control Flow Integrity, which might be violated by PipeWire calls, because of the way the PipeWire library is used (it is dlopened). We had the same problem in the past with screencast support. This was uncaught and released, but I fixed it already in time to be backported to M128, which is behind the doors already. Chromium will also benefit from all of the above fixes because it’s all done in WebRTC, which is shared with Chromium and Firefox.

Testing PipeWire camera support

Testing is something we need right now to catch all the bugs in time for the Fedora 41 release. In the case of Chromium, you will need to wait for Chromium 128 and enable the flag as mentioned above. For Firefox you can go to about:config and enable the media.webrtc.camera.allow-pipewire option. In the case of Fedora 41+, you shouldn’t need to do this as we’ve already made the switch in the latest Firefox build. You can also read my previous blog post for more details, but you can ignore all the issues there as they are already fixed. If you do find a problem, the best place to report it is on the WebRTC bug tracker, as it’s most likely a general issue, not specific to Chromium or Firefox.

In most setups (default), you will probably end up using PipeWire camera with V4L2 backend in PipeWire, but you can also install libcamera and libcamera plugin in PipeWire as mentioned in my previous blog post. The only thing that has changed is that recent Wireplumber will avoid duplicate camera entries and libcamera nodes will be hidden. To filter out V4L2 nodes and show libcamera instead, you can create a .config/wireplumber/wireplumber.conf.d/disable-v4l2.conf file with the following content:

wireplumber.profiles = {
  main = {
    monitor.v4l2 = disabled
  }
}

After restarting Wireplumber, you should see only libcamera nodes. You can check this with wpctl status.

Now let’s make Fedora 41 another great release, packed with the latest technologies.

Although Wayland screen sharing is still not yet enabled by default in Chromium, which is what I hoped to achieve this year, I think I can say we are almost there and you can expect it sooner than later. Let’s summarize what we have accomplished this year to make this change happen:

Stream restoration support

You probably remember that you had to go through two portal (xdg-desktop-portal) dialogs all the time you wanted to share your screen. We had first portal dialog to have your selected screen visible in Chromium preview dialog and yet another portal dialog to make your screen shared with the web page itself. This was quite annoying as users had to make the same selection twice. Thanks to a new addition into portal API I was able to implement stream restoration support in WebRTC to bypass the second portal dialog and have your selection instantly shared with the web page itself once you confirm it in the Chromium dialog. This was released in Chromium 105.

Tests for PipeWire (streaming) code

This is not a feature that is visible to users, but it makes an important part of the whole process. It was a long effort to make this happen as it’s something that is not trivial to test and needs some dependencies for the tests itself to run. As a first step we had to bring PipeWire and some of its dependencies into the infrastructure. As easy as it sounds, in order to add a new dependency there you have to add it in form of a CIPD (Chrome Infrastructure Package Deployment) package. This means you write recipes with information how to build and get your package. This all on a CentOS 7 based distribution where you have to work with older libraries or missing libraries and tools (e.g. Meson). This makes your packages later available in third_party directory that is available to both Chromium and WebRTC. Next step was to write the tests itself. The only way how I could test our PipeWire code, code that is all about receiving frames over PipeWire stream, was to write another “testing” stream that will be sending us frames with parameters where we will know what to expect and can verify we received what we were supposed to receive. For the tests itself I used GTest framework which is very well documented and quite comprehensive. Sadly, we were still just in the middle of the process and one would hope that it cannot get more complicated, but the opposite is true. There is a whole runtime setup we need for our tests to run. We need PipeWire and PipeWire session manager to run, otherwise we would never activate and connect our streams. To create the setup I had to write a Python wrapper script that sets all environment variables for PipeWire to find everything in the third_party directory (plugins, libspa, etc.) and run both PipeWire and PipeWire session manager in order to run our test. As a last step, because we had a script that runs the test, we had to create a mapping in the infrastructure making the script itself a launcher of our test + limit this only to x86_64 architecture as that’s the only one where we have PipeWire available as CIPD package.

Here are upstream changes that implement all above mentioned:

UX improvements in Chromium preview dialog

All these improvements were made by Alexander Cooper, Alex is Google engineer working on screen sharing stack in Chromium and WebRTC. I have been intensively working with Alex for the past year and he is my go to person when it comes to code reviews or anything related to screen sharing in Chromium. Alex made a great set of UX improvements in their preview dialog. My original implementation always automatically invoked portal dialog when screen sharing was initiated. This led into one dialog overlaping the other. In most recent Chromium version (I think starting with 107) users will be presented with Chromium preview dialog first asking to share a web tab and only once they pick to share a screen they will get presented with the portal dialog. You can also now re-request the portal dialog in case you pick a wrong screen to share.

Web Engines Hackfest in A Coruña

I travelled to A Coruña in May to attend the Web Engines hackfest and meet Alex Cooper from Google there. This was a perfect opportunity for us to meet in person and I’m really grateful for that. Even though that for me it was a bit stepping outside of my comfort zone as I went somewhere where I didn’t know anyone (besides Alex), but as I later found out, all the people there were super friendly and I’m happy that I met some new faces. We had very productive conversations with Alex and having enough time to talk in person was beneficial for both sides as we could explain to each other technical details of our backgrounds and talk about future plans and current issues.

Firefox and WebRTC rebase

Firefox upstream has been behind with our WebRTC changes for more than ~2 years. This has changed recently with Firefox 106, where they finally rebased their WebRTC version to M103 (Chromium 103). I also provided them list of additional backports they should pick up in order to have fixes for some crashes and issues we have fixed since then. Sadly, as I later found out, even though they did the rebase and all the backports, the new codebase (for screen sharing) is not in use and instead they still keep the old code and use it instead. This is because they don’t have all the dependencies in place and it’s been blocked on this issue since then, hopefully it will get figured out soon and also Firefox users can benefit from all the improvements we did over the past two years. This is not an issue for Fedora users where I created a patch for our Firefox package that enables the new code and use it instead.

Plans for the future

Extend PipeWire code test coverage. Currently we test only the essential part of the code, but I would like to further extend the tests with tests for all kinds of metadata we can use (damage regions, crops, mouse cursor etc.)
Use portal dialog as the default one. This has some requirements that need to be fulfilled before Chromium can rely on it. We need to show screen/window previous, as currently it can easily happen you pick a wrong screen (especially when you have two identical monitors) and we need a way how to invoke Chromium dialog so users can share a web tab.
Fix bugs. I’m currently not aware of any issue and we already fixed a bunch of them this year as people use it more often, but I expect more issues to appear as Wayland becomes more and more dominant and we finally make it enabled by default.

Jan Grulich

Fedora, KDE, Qt, C++, QML

Making PipeWire default option for Firefox camera handling