Jan Grulich

xdg-desktop-portal

When Your Webcam Doesn’t Work: Solving Firefox and PipeWire Issues

- jgrulich Leave a comment

If you are a regular reader of my blog posts, as I am sure you are, you will know that we made a switch with Fedora 41 and now use PipeWire as the default backend for camera handling in Firefox. It won’t come as a surprise that such a huge change is not without its problems. After talking to many of you and debugging the same issues over and over again, I would like to go through most of the common issues and show you how to fix them, and also shed some light on the whole stack.

For Chrome/Chromium users out there, most of the issues mentioned here also apply to you, as most of the PipeWire camera code is shared in WebRTC, but I must mention that the PipeWire camera is completely broken in M131 and fixed in M132.

Issue #1 – Permission issue – “Camera is blocked” or “The request is not allowed” etc.

This is the most common problem and is usually caused by the user not giving access to the camera to an application that is requesting it. When Firefox wants to use a camera, it makes a request to the camera portal (xdg-desktop-portal). This results in a system dialogue asking for camera access for Firefox. If access is granted, it will be granted for all future sessions, but if access is denied or the dialogue is closed, it will remember this decision and all future requests to use the camera will be automatically denied.

You can check this by running:

$ flatpak permissions devices camera
Table   Object App                 Permissions Data
devices camera                     yes         0x00
devices camera org.mozilla.firefox yes         0x00

In the result you will see that “org.mozilla.firefox” has “yes” stored in the permission store. There is also an empty entry with “yes” stored. The empty entry is usually for applications for which we were unable to get an application id. This happens for host applications that are launched in an unusual way, such as the Alt + F2 command or from a terminal. If you have a permission problem, you will most likely see “no” stored there, and this is what is causing the problem for you.

You can clear this and be prompted again for camera access running this command:

flatpak permission-remove devices camera org.mozilla.firefox

You may be wondering why flatpak is involved, since you don’t use flatpak applications. Flatpak is not really necessary, but I use its command line to work with the permission store and it is easier for me to just give you a command and you give me the result, otherwise you could also use Flatseal to check your camera permissions. The permission store comes from portals (xdg-desktop-portal), which we use to get access to your camera. While portals were originally intended to be used mainly by sandboxed applications (Flatpak, Snap), they are now also used for things like screen sharing (Wayland) and now the PipeWire camera, making them an essential part of the Linux desktop stack. Always make sure that xdg-desktop-portal is installed with a specific portal backend for your desktop, e.g. xdg-desktop-portal-gnome for GNOME or xdg-desktop-portal-kde for Plasma.

Issue #2 – No camera found

This can be a problem with many components. Let’s start with the most important one, which is finding out if Wireplumber (the session and policy manager for PipeWire) detects it.

You can run:

$ wpctl status
Video
 ├─ Devices:
 │      50. Integrated Camera                   [v4l2]
 │      62. Integrated Camera: Integrated C     [libcamera]
 │      63. Integrated Camera: Integrated I     [libcamera]
 │      69. Integrated Camera                   [v4l2]
 │      85. Integrated Camera                   [v4l2]
 │      93. Integrated Camera                   [v4l2]
 │  
 ├─ Sinks:
 │  
 ├─ Sources:
 │      76. Integrated Camera (V4L2)           
 │  *   80. Integrated Camera (V4L2)           
 │  
 ├─ Filters:
 │  
 └─ Streams:

Here we are mainly interested in “Sources“, as this is what will appear in Firefox. Typically, most laptop cameras appear here twice, as one is an infrared camera for Windows Hello support, which we already filter out in Firefox. If your camera doesn’t appear there, it won’t work in Firefox or any other application that uses PipeWire.

If your camera is listed there but doesn’t appear in Firefox, I usually recommend that people try OBS Studio, which has great support for PipeWire cameras. This will always tell you if the problem is in Firefox or somewhere else. If it works in OBS Studio, you can open a bug to Firefox with all the necessary information (see below). If not, it is probably a bug in PipeWire.

We are already tracking one issue with the v4l2 plugin in PipeWire. This is most likely a race condition for which we have at least a workaround in the form of switching from v4l2 to libcamera.

In order to use libcamera, you can create following file:

$HOME/.config/wireplumber/wireplumber.conf.d/99-libcamera.conf

With the following content:

wireplumber.profiles = {
 main = {
   monitor.v4l2 = disabled
   monitor.libcamera = optional
  }
}

And restart both Wireplumber and PipeWire:

systemctl --user restart pipewire wireplumber

If this doesn’t solve the problem for you, please follow the instructions below to report a bug to Firefox.

Another known problem, probably a rare one, is if you restart PipeWire while Firefox is still running. This is because we keep a connection to PipeWire and when you restart it, that connection is broken and not initialised again. This problem affects OBS Studio in the same way and I’m already working on a fix. The solution here is to restart Firefox.

Debugging and reporting issues to Firefox

You came here because none of the above worked? You can still report a bug with all the necessary information to help us identify the problem. First, you want to report a bug to Firefox upstream. You can do this here by selecting the “Core” product and the “WebRTC: Audio/Video” component and providing all the logs from below.

Include DBus communication with xdg-desktop-portal.

Open a terminal of your choice and run:

dbus-monitor --session

Keep it running, while you try to access the camera in Firefox. For example, using the WebRTC getUserMedia test page. You should see all the DBus communication in the log from dbus-monitor.

Also a useful information might be to know whether the camera portal see any camera by running:

dbus-send --print-reply --dest=org.freedesktop.portal.Desktop /org/freedesktop/portal/desktop org.freedesktop.DBus.Properties.Get string:"org.freedesktop.portal.Camera" string:"IsCameraPresent"

Including log from Firefox by running it with:

MOZ_LOG="MediaManager:5,CamerasParent:5,CamerasChild:5,VideoEngine:5,webrtc_trace:5"

And a log from:

pw-mon

Which will provide all advertised formats by your camera and supported formats by Firefox.

Last resort

Once you have done your duty and opened a bug with all the above information, which I’m really grateful for, you can now go to “about:config” in Firefox, disable “media.webrtc.camera.allow-pipewire” and restart Firefox. This will switch back from PipeWire to v4l2, but I hope you will accept this as a temporary solution until we can identify and fix your problem.

Debugging and reporting issues to Chromium

All the logs you can provide also apply to Chrome/Chromium, with the exception to logs from the app itself.

In order to get logs from Chrome/Chromium, you need to run it with:

google-chrome --enable-logging --vmodule=*/webrtc/*=1

Once you have collected all the necessary logs, you can open a bug here for the “CameraCapture” component and add me to the bug (use grulja AT gmail.com) or let me know at least so I’m aware.

To switch back from PipeWire to v4l2 you have to go to “chrome://flags” and disable “PipeWire Camera support“, but you already know that since you had to enable it yourself before :).

WebRTC: journey to make wayland screen sharing enabled by default

- jgrulich 12 Comments

While we have pretty good support for screen sharing on Wayland in WebRTC, which is included in browsers like Chromium or Firefox, it is still not enabled by default in Chromium and it is kept behind a flag. Not only you have to remember to always enable it for new configurations, but for many users it is not even something they are aware of. This has been my main focus recently and I would like to share with you steps that has been done and what are the plans for the future.

What are the changes to expect in Chromium soon?

DMA-BUF improvements/fixes:

Last year I landed proper DMA-BUF support in WebRTC, which made things way faster. It was working, but it was not perfect and there were some corner cases where it might not be working at all. Here are changes I made recently:

  • Advertise DMA-BUF support when it is really supported. Older versions of PipeWire don’t handle the new way of DMA-BUF negotiation and therefore it shouldn’t be used in such cases. Also using DMA-BUF modifiers requires some recent versions of PipeWire on both sides.
  • Implemented stream renegotiation. In situation when we fail to import a DMA-BUF with given modifier, we will drop this modifier and try to renegotiate stream parameters and go with a different modifier or fallback to shared memory buffers in case we fail completely.
  • Make sure to import DMA-BUF with correct render node. In case of multi-gpu setups, we always picked the first render node to import DMA-BUFs, but it can happen that they were actually produced by a different render node and for that reason we might fail to import them. We now try to get default EGLDisplay, which should be the same one used by the wayland compositor and we should be using same render node.

Better mouse cursor support:

Until now we had mouse cursor as part of the screen content. This means that everytime you moved with your mouse cursor, we had to update whole image and that is very inefficient. The API in WebRTC allows you to implement MouseCursorMonitor which can be used to track mouse changes only and each platform can have both MouseCursorMonitor and DesktopCapturer implementations combined in DesktopAndCursorComposer to get complete image and this all works automatically like a magic. Unlike X11 implementation, our only option is to get everything from one PipeWire stream we connect to and there was no way how to make it shared from DesktopCapturer implementation so it can be used by MouseCursorMonitor implementation. I had to split DesktopCapturer to have xdg-desktop-portal and PipeWire separate implementations. Code for PipeWire is now a SharedScreenCastStream class which is being shared through DesktopCaptureOptions. This is set of parameters associated with each capturer instance and luckily this is also passed to MouseCursorMonitor so we can have access to already initialized PipeWire stream and get the cursor data from there. Implement MouseCursorMonitor with SharedScreenCastStream was then piece of cake.

List of merge requests:

This should again significantly improve performance of screen sharing, because moving with a mouse over a static screen content doesn’t need full screen content update.

Misc:

Last but not least, I’m now in touch with Google developers who help me to review all my changes and discuss with me the current state, issues I have, etc. on monthly meetings we have. The plan is to make this finally enabled by default, hopefully in the first half of this year. There are still some things that need to be solved before this is enabled and there is lot of work ahead, but things look promising.

Plans for the future:

  • Implement stream restoration
    • this will allow us to skip the second portal dialog and I already have plan in my head how to do this in WebRTC. This is currently only supported by xdg-desktop-portal-gnome and xdg-desktop-portal-kde lacks this functionality.
  • Improve UX of the Chromium screen sharing dialog
  • Write tests for all PipeWire/portal code in WebRTC

Even though WebRTC is used in Firefox, I mostly talk about Chromium, because Firefox doesn’t use most recent WebRTC and will need to pick all the changes I did or rebase to newer WebRTC in order to have them. Firefox also has PipeWire/Wayland screen sharing enabled by default and doesn’t have UX issues as there is no internal screen sharing dialog like in Chromium.

I hope all these changes will make your experience better and next time when you read a new blog post I will be informing you about end of this journey.

How to use libportal/libportal-qt

- jgrulich Leave a comment

There was a blog post from Peter Hutterer about Flatpak portals posted few months back. Peter explained what are portals and how do they work. Portals are used mostly because of security and sandbox/Wayland restrictions. Many times your only way to get access outside (opening a file, sending a notification, sharing a screen, etc.) is to use a portal. For most use-cases applications or developers don’t need to care about them as their support is usually implemented in libraries they use. For example Qt and GTK use portals internally so apps can use still the same APIs as before and they don’t need to worry about their apps not working in sandboxed environments. BUT there are still scenarios where libraries have unsufficient or none portal support, or a different options are desired so what are the options in this case if you still need to use portals?

  1. Do everything yourself, which means you will implement all the DBus calls and handling yourself.
  2. Use a library. Most logic choice would be libportal, but there is also a project called ASHPD for Rust users.

What is libportal and libportal-qt?

The libportal library provides GIO-style async APIs for Flatpak portals. It hides all the DBus complexity users would face in case of using portals directly and provides a user-friendly library instead. You might think that the libportal-qt is the same thing, just with Qt-style APIs, but the idea behind it is that each toolkit (Gtk3, Gtk4, Qt5, Qt6) has a different way to get a window handle which is needed to associate portal dialogs with the app that invoked them. So libportal-qt just provides a way to get a XdpParent object from a QWindow. As a C++/Qt developer I don’t mind using C/Glib APIs and I used it many times, but there is still one speciality I fail to use everytime, my friend GVariant. Some of the portal APIs in libportal expects a GVariant for all the complex structures, for example to specify a filter option for OpenFile() call from the fillechooser portal, you have to build a very complex GVariant based on the DBus specification.

Remember I told you libportal-qt doesn’t offer Qt-style APIs? This is not necessarily true, because I implemented all the complex structures you will have to pass in most of the portals and implemented functions that will return them as GVariants so you don’t need to get in touch with GVariants at all.

How to use libportal-qt?

First of all, all libportal flavours have pkgconfig file installed so it’s easy to use them from any build system and you just need to search for libportal-qt5 (we don’t have -qt6 version yet).

And how does the code look like? For example let’s say you want to open an image:

// Creates a filter rule, this can be a Mimetype or Pattern.
XdpQt::FileChooserFilterRule rule;
rule.type = XdpQt::FileChooserFilterRuleType::Mimetype;
rule.rule = QStringLiteral("image/jpeg");

// Create a filter with our rules, we will then pass it to OpenFile() call as GVariant.
XdpQt::FileChooserFilter filter;
filter.label = QStringLiteral("Images");
filter.rules << rule;

// Create a GVariant from our filter. This will result into variant in form of:
// "[('Images', [(1, 'image/jpeg')])]"
g_autoptr(GVariant) filterVariant = XdpQt::filechooserFiltersToGVariant({filter});

// Get XdpParent to associate this call (portal dialog) with our window.
XdpParent *parent = xdp_parent_new_qt(m_mainWindow->windowHandle());

// Finally open a file. XdpQt::globalPortalObject() is another convenient function 
// that creates a global instance of XdpPortal object so you don't need to take care
// of creating it yourself. For some of the arguments we just pass a nullptr to don't 
// specify them.
xdp_portal_open_file(XdpQt::globalPortalObject() /*XdpPortal object*/,
                                  parent /*XdpParent object*/, "Title", filterVariant /*filters*/,
                                  nullptr /*current_filter*/, nullptr /*choices*/, 
                                  XDP_OPEN_FILE_FLAG_NONE /*flags*/, nullptr /*cancellable*/, 
                                  openedFile /*callback*/, this /*data*/);
xdp_parent_free(parent);

// Then the callback would look like this, eg.
static void openedFile(GObject *object, GAsyncResult *result, gpointer data) {
    g_autoptr(GError) error;
    g_autoptr(GVariant) ret = 
        xdp_portal_open_file_finish(XdpQt::globalPortalObject(), result, &error);

    if (ret) {
        // Another convenient function that will get you uris and choices from 
        // GVariant returned by xdp_portal_open_file() call.
        XdpQt::FileChooserResult result = filechooserResultFromGVariant(ret);
        
        // Do whatever you want to do with the result. Here we just print opened selected files.
        qDebug() << result.uris;
    }
}

As you can see, no GVariant got hurt and you can easily open a file without any GVariant knowledge. Besides FileChooser portal helpers, we also have Notification portal helpers, because serializing icons and buttons is also something that is not trivial. For the rest of the portals you either don’t need to use complex GVariants so you can use them easily without helper functions same way as shown above, or some portals like ScreenCast or RemoteDesktop are not used that often and we don’t have helper functions for those just yet.

I hope you can find this helpful in case you want to join this world. The libportal project is hosted on GitHub in case you want to try it just now, because this is still not part of any stable release (will be in libportal 0.6), or report a bug or just look at my GVariant helpers to see what I spare you of.

WebRTC/Chromium updates in 2020

- jgrulich 10 Comments

In 2019, I started with my first contribution to WebRTC. This was all about screen sharing support on Linux Wayland sessions, using xdg-desktop-portal and PipeWire. Back then, it was quite simple, we only had PipeWire 0.2 and all portal backends supported only screen sharing (no window sharing). While this was relatively easy, it was not ideal as each screen sharing request involved two portal dialogs to get the screen content on the web page itself. For me it was a big success, because I made quite a significant contribution to such a big project, which is used by many people, and a project which is used by all modern web browsers.

At the beginning of 2020, the year everyone would like to erase from their memories, we got PipeWire 0.3 (with slightly different API) and later with xdg-desktop-portal-gtk and xdg-desktop-portal-kde (later this year) people were finally able to share application windows. Support for all of this was lacking in WebRTC, because back then those were not available. I wanted to tackle all issues at once, bring support for window sharing and get rid of the “dialog hell” with portals, which was even worse with the new window sharing capabilities in portal backends.

This is what the situation looks like. With each request to share a screen, you got the preview dialog from Chromium. This dialog consists from three pages. One is for screen sharing making one portal request, second one is for window sharing, which is another portal request, and the last one is just to allow you to share a web page you have opened. You had to confirm both portal dialogs, then confirm the Chromium dialog and finally you got one more portal dialog (ouch) to get the screen content on the web page itself.

I had a solution. I made all portal calls identified with an ID and shared this ID (portal call) in Chromium between both pages in the Chromium preview dialog and with the request made for the web page itself. With this solution we only had ONE portal dialog. This was a perfect solution (at least seemed to be). I started working on this at the beginning of this year, we exchanged many emails with people from Chromium UX team, because I wanted to do also some minor UI changes in the preview dialog. Unfortunately, those were rejected for consistency with all platforms. It was not a big deal and I submitted my changes for review, keeping UI as it was, just adding all necessary bits into Chromium and WebRTC to make it all work.

I wish to say things went smoothly since then, but the opposite is true. It took a while to get everything reviewed, but this is probably no surprise with this year being weird and many people working from home with less than ideal conditions. Anyway, few months passed away, I ended up rewriting my changes many times, not even counting hours I spent on it. This all resulted into me being obsessed with this change, it mattered to me so much to get it merged. I was constantly thinking about how to make it better, I was many times fixing issues in the evening (as reviewers were mostly US based), instead spending time with my family. It would be even better to waste my time with my beloved Playstation. This had really negative impact on my mental health and I realized this has to stop and I simply gave up, because I couldn’t continue this way and needed a break. I abandoned both changes (WebRTC and Chromium) and decided to just pick changes I will be able to successfuly upstream. I probably made my change too ambitious and complicated or maybe it’s just Chromium not being ready for this kind of change, because some tweaks were specific for my use-case. It’s also hard to say I wish upstream devs had helped me more, because there is so much to understand around Wayland, portals and PipeWire and way how it all works together.

Anyway, with a new start, without pressure after gaving up on the change, I picked the most important changes and submitted them separately. I was surprised now how smoothly this went and how fast those changes were upstreamed. Simply those changes were simple, understandable and easy to review. I didn’t gave up on fixing the “dialog hell” completely, I have some other ideas, but next time I will try to submit them step by step and will keep some distance and my free time.

And what are the changes you can expect in upcoming Chromium release in 2021?

Support for PipeWire 0.3

You can now build Chromium/WebRTC with both PipeWire 0.2 and Pipewire 0.3. There is a new “rtc_pipewire_version” option you can pass to your build configs.

Window sharing support

There is probably no description needed. You will be able to share application windows in case you don’t want to share whole screen.

Suppport for DmaBuf and MemFd buffer types

This should allow faster transfer of your screen content from your Wayland compositor, through PipeWire to your browser.

Less portal dialogs involved

If you look back into the screenshot I posted above, you can see there are two portal dialogs opened just for the Chromium preview dialog. I at least tried to reduce this to just one portal dialog. This was done by removing the page for window sharing, because the screen share request will already handle both screen and windows.

I think you can expect above mentioned changes in Chromium 89 and I hope you will at least appreciate some of these improvements even though I didn’t deliver everything I wanted to. Also, thanks to Martin Stránský from our Firefox team, you can expect all these changes to be also part of Firefox.

Happy holidays and see you in a better year.

Scroll To Top