Chromium Wayland HiDPI support

>[!warning]- Info > Author: [email protected] > Date: 2024/05/05 > Status: WIP > Revision: 0 > Visibility: Public ## Fractional Scaling ### Current Chromium implementation - Architecturally, fractional scale has been assumed to be a per-display info, which is represented by [display::Display](https://source.chromium.org/chromium/chromium/src/+/main:ui/display/display.h;l=34;drc=071a85757d9d95316f83f50e1fe087a6d87908f9) instances, which is not the case for fractional scale under Wayland. - To work around such protocol limitation / design decision, Chromium infers it out of xdg-output's logical size, which seems a consensus among Wayland and compositor developers to be a misuse of that protocol and shouldn't be done. - Issues have been observed in the wild, such as [rounding differences](https://issues.chromium.org/issues/40279238), leading to per-compositor [quirks](https://chromium-review.googlesource.com/c/chromium/src/+/5404799) proposals, etc. - Another problematic assumption taken in Chromium, in the context of Wayland, is that the display server provides information about in which display a surface (predominantly) is. - Such information is not available and [current implementation](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=263-290;drc=2313c7b1eac8a865df914f036ae10de3b4607dfa) returns the entered output with the largest scale factor (sigh!). ^c00a68 - For Wayland, display scale "calculation" currently relies solely on `xdg-output` protocol extension, there's no notion of "window's preferred scale", so `wp_fractional_scale.preferred_buffer_scale` is basically ignored. - For platform-agnostic code to determine the scale that must be used for a given window, `display::Screen` API is used to figure out the display on which a window predominantly is (which is unsupported under Wayland, see the item [[#^c00a68|above]]), and that display's scale is used; - The no-op impl for that event was introduced [here](https://chromium-review.googlesource.com/c/chromium/src/+/4370091/21..28/ui/ozone/platform/wayland/host/wayland_surface.cc#905) along with its motivation. - Recent crbug filed to properly support `preferred_buffer_scale`: https://issues.chromium.org/issues/336007385 - Initial attempt from Thomas (later reverted): https://crrev.com/c/4322839 - Blurring issues observed (at least) in Kwin. Workaround [proposed](https://chromium-review.googlesource.com/c/chromium/src/+/5513723). Double-check, but `wp-fractional-scale` should likely be the long-term and correct solution. TODO: Check current status. - Fractional scaling is currently enabled by default, but can be disabled using the `WaylandFractionalScaleV1` [feature](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/common/features.cc;l=31;drc=4c2d0f00bc87614ef325db3811efa2a905a15482) ### Relevant protocols #### wl_surface.preferred_buffer_scale event - Landed on Feb 2023. version 6 - MR: [protocol: add wl\_surface.preferred\_buffer\_scale and transform (!220) · Merge requests · wayland / wayland · GitLab](https://gitlab.freedesktop.org/wayland/wayland/-/merge_requests/220) - Motivation: Make scale update independent from wl_output events. Supports non-fractional cases only. #### wp-fractional-scale - Landed on Nov 2022 - MR: [wp-fractional-scale-v1: New protocol for fractional scaling (!143) · Merge requests · wayland / wayland-protocols · GitLab](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/143) - Similar to `wl_surface.preferred_buffer_scale` but for fractional scaling. #### Affected Web APIs [Window: devicePixelRatio property - Web APIs | MDN](https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio) > The **`devicePixelRatio`** of [`Window`](https://developer.mozilla.org/en-US/docs/Web/API/Window) interface returns the ratio of the resolution in _physical pixels_ to the resolution in _CSS pixels_ for the current display device. Status: - Stable API - Supported by basically [all major browsers](https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio#browser_compatibility) [ScreenDetailed: devicePixelRatio property - Web APIs | MDN](https://developer.mozilla.org/en-US/docs/Web/API/ScreenDetailed/devicePixelRatio) > The **`devicePixelRatio`** read-only property of the [`ScreenDetailed`](https://developer.mozilla.org/en-US/docs/Web/API/ScreenDetailed) interface is a number representing the screen's device pixel ratio. Status: - Experimental - Supported only in [Chromium-based browsers](https://developer.mozilla.org/en-US/docs/Web/API/ScreenDetailed/devicePixelRatio#browser_compatibility) _**Side-note:** For Lacros, specific protocols are used, such as aura-shell, surface-augmenter, wp-viewporter._ #### Per-window preferred scale An detailed analysis and proposal is in progress and available at [[Support platform-controlled window-driven scaling in Chromium]]. ##### Relevant community discussions - [Deprecate parts of wl\_output and xdg\_output (#458) · Issues · wayland / wayland · GitLab](https://gitlab.freedesktop.org/wayland/wayland/-/issues/458) - [Incorrect xdg\_output logical size for scaled outputs (#2631) · Issues · GNOME / mutter · GitLab](https://gitlab.gnome.org/GNOME/mutter/-/issues/2631) - [Draft: fractional-scale-v1: Add support for binding to the output (!303) · Merge requests · wayland / wayland-protocols · GitLab](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/303) - [idle-notify: new protocol (!29) · Merge requests · wayland / wayland-protocols · GitLab](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/29) ##### Other resources - Blog post by David Edmundson about fractional scales on kwin-wayland | [My month in KWin/Wayland – David Edmundson's Web Log](https://blog.davidedmundson.co.uk/blog/my-month-in-kwinwayland/) - Blog post by David Edmundson about kwin coordinates system and scaling | [KWin Wayland Independent Monitor Scaling – David Edmundson's Web Log](https://blog.davidedmundson.co.uk/blog/kwin_and_scaling/) - Alex's blog post about HiDPi support in Chromium/Wayland (2020) | [HiDPI support in Chromium for Wayland | Alex likes to code](https://blogs.igalia.com/adunaev/2020/11/13/hidpi-support-in-chromium-for-wayland/) --- ## GTK text scaling Issues: 1. [UI issue with gnome zoom level\>1, gtk=4, and refresh desktop design \[41486578\] - Chromium](https://issues.chromium.org/issues/41486578) - Specific about omnibar text visibility and positioning 1. [Gnome accessibility's 'Large Text' feature is not applied on wayland \[40856031\] - Chromium](https://issues.chromium.org/issues/40856031) - Broader. About whether Chrome respects and how does it handle the text scale factor setting. - Involves a set of features currently unsupported/broken: - Rendering. Text and UI elements partially or not scaled at all. - On-the-fly visual updates. UI does not get updated as soon as the setting is changed (works on X11). Browser restart is needed instead. - Text scale factor can be set, for example, via [Gnome Tweaks]() app - Chromium X11 backend works seamlessly ### X11 vs Wayland - [WindowTreeHost::OnDisplayMetricsChanged](https://source.chromium.org/chromium/chromium/src/+/main:ui/aura/window_tree_host.cc;l=651-655;drc=41374c974d98f8cf67134f9ddb8d96d398154dfe) is the relevant entry point for the Ozone/X11 path, which updates UI on-the-fly upon GTK's font scale changes - It is triggered by XDisplayManager's display list update, which in turn can be reached from 2 different paths: - [GtkUi::OnGtkXftDpiChanged](https://source.chromium.org/chromium/chromium/src/+/main:ui/gtk/gtk_ui.cc;l=760;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) > [GtkUi::UpdateDeviceScaleFactor](https://source.chromium.org/chromium/chromium/src/+/main:ui/gtk/gtk_ui.cc;l=1044;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) > [X11ScreenOzone::OnDeviceScaleFactorChanged](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/x11/x11_screen_ozone.cc;l=265;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) - [XDisplayManager::OnEvent](https://source.chromium.org/chromium/chromium/src/+/main:ui/base/x/x11_display_manager.cc;l=74;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) > [X11ScreenOzone::OnDeviceScaleFactorChanged](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/x11/x11_screen_ozone.cc;l=265;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) - Which means that Xft DPI events imply in actual display scale changes. Which is notified through XRandR's notifyevent - TODO: Confirm in XRandR docs and test on KWin too? - That's not the case on Wayland. - For the Wayland backend, it is also reached, though: - It might be buggy as it depends on Screen::GetDisplayNearestWindow, which is [buggy](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=282-304;drc=b7979b7a71fa422fd46d72f6c4a6efd6bd121863) under Wayland. For most use cases, eg: window fully located in a single display, it works as expected. - Sub-tasks: - [x] Make sure browser UI can be inspected with [UI devtools](https://chromium.googlesource.com/chromium/src/+/main/docs/ui/learn/ui_debugging.md) - [x] Does the window bounds change after the update? - dip vs physical px - Yes. See the views hierarchy logs below. - [x] Does re-layout actually happen (or is it only re-scaling) ? - Yes. Bounds/scale change trigger views relayout in the process of producing a new frame. - TODO: More in-depth explanation not needed for now as it's clear that's not the main problem. - [x] What does need to be done at WTHPlatform/platform window level to prevent the `wp_viewport.set_destinaion` problem? - WaylandWindow's surface is not aware of the scale change in course? - OOC: Why isn't that a problem in X11? Maybe there's no such notion from the X Server perspective, and it deals only with physical pixel coordinates? Commands: ```bash # Create a few empty dirs to be opened as tabs in chromium mkdir -pv $(seq -f "/tmp/%04.0f" 0 4) # Launch chrome with relevant debug args autoninja -C out/linux chrome && \ chr_run --enable-features=ui-debug-tools --enable-ui-devtools \ /tmp/000* --gtk-version=4 --ozone-platform=<x11 or wayland> 2>&1 | \ grep -e 'ERROR\|VERBOSE\|<$BrowserRootView\|Label\|TabStrip\|Tab$ ' ``` #### Ozone/X11 ##### Font scale 1.0: ![[Pasted image 20240808150941.png]] Views hierarchy: ``` <BrowserRootView bounds="0 0 584x320" enabled="true" id="0" needs-layout="false" visible="true"> <Label bounds="0 0 0x0" enabled="true" id="6" needs-layout="true" visible="false" /> <TabStrip bounds="28 0 316x41" enabled="true" id="21" needs-layout="false" visible="true"> <Tab bounds="0 0 93x41" enabled="true" id="20" needs-layout="true" visible="true"> <Label bounds="44 12 5x16" enabled="true" id="0" needs-layout="true" visible="true" /> <Tab bounds="75 0 93x41" enabled="true" id="20" needs-layout="true" visible="true"> <Label bounds="44 12 29x16" enabled="true" id="0" needs-layout="true" visible="true" /> <Tab bounds="150 0 92x41" enabled="true" id="20" needs-layout="true" visible="true"> <Label bounds="44 12 28x16" enabled="true" id="0" needs-layout="true" visible="true" /> <Tab bounds="224 0 92x41" enabled="true" id="20" needs-layout="true" visible="true"> <Label bounds="44 12 28x16" enabled="true" id="0" needs-layout="true" visible="true" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> ``` _** Focus on root, tabs and label items_ Debug logs when switching to 1.25 scale: ``` [323306:323306:0807/182451.063975:ERROR:gtk_ui.cc(770)] OnGtkXftDpiChanged -------------------------------- [323306:323306:0807/182451.064007:ERROR:gtk_ui.cc(1059)] UpdateDeviceScaleFactor display_config changed. Updating and notifying. font_scale=1.25 [323306:323306:0807/182451.125436:ERROR:window_tree_host.cc(657)] OnDisplayMetricsChanged ==================== [323306:323306:0807/182451.125461:ERROR:window_tree_host.cc(665)] OnDisplayMetricsChanged ==================== will resize! [323306:323306:0807/182451.125469:ERROR:window_tree_host.cc(604)] OnHostResizedInPixels new_size_in_pixels=532x316 prev_scale=1 new_scale=1.25 bounds=0,0 532x316 [323306:323306:0807/182451.125488:ERROR:desktop_window_tree_host_platform.cc(898)] CalculateRootWindowBounds size_in_dip=427x253 [323306:323306:0807/182451.125494:ERROR:window_tree_host.cc(400)] UpdateRootWindowSize new_bounds=0,0 427x253 [323306:323306:0807/182451.125499:ERROR:window.cc(469)] SetBounds 2 BrowserFrame [323306:323306:0807/182451.125632:ERROR:compositor.cc(463)] SetScaleAndSize scale=1.25 size_in_pixel=532x316 [323306:323306:0807/182451.135990:ERROR:window.cc(469)] SetBounds 2 DesktopNativeWidgetAura - content window [323306:323306:0807/182451.146314:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [323306:323306:0807/182451.146391:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [323306:323306:0807/182451.146406:ERROR:window.cc(469)] SetBounds 2 RenderWidgetHostViewAura [323306:323306:0807/182451.146785:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [323306:323306:0807/182451.146792:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [323306:323306:0807/182451.146831:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [323306:323306:0807/182451.146836:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [323306:323306:0807/182451.146881:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [323306:323306:0807/182451.146886:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [323306:323306:0807/182451.148500:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [323306:323306:0807/182451.148511:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura ``` ##### Font scale 1.25: ![[Pasted image 20240808151336.png]] Views hierarchy: ``` <BrowserRootView bounds="0 0 468x257" enabled="true" id="0" needs-layout="false" visible="true"> <Label bounds="0 0 0x0" enabled="true" id="6" needs-layout="true" visible="false" /> <TabStrip bounds="28 0 200x41" enabled="true" id="21" needs-layout="false" visible="true"> <Tab bounds="0 0 64x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="20 12 0x16" enabled="true" id="0" needs-layout="false" visible="false" /> <Tab bounds="46 0 64x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 0x16" enabled="true" id="0" needs-layout="false" visible="false" /> <Tab bounds="92 0 63x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 0x16" enabled="true" id="0" needs-layout="false" visible="false" /> <Tab bounds="137 0 63x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 0x16" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="16 14 356x23" enabled="true" id="0" needs-layout="false" visible="true" /> ``` #### Ozone/Wayland ##### Font scale 1.0: ![[Pasted image 20240808154948.png]] Views hierarchy: ``` <BrowserRootView bounds="0 0 584x320" enabled="true" id="0" needs-layout="false" visible="true"> <Label bounds="0 0 0x0" enabled="true" id="6" needs-layout="true" visible="false" /> <TabStrip bounds="28 0 343x41" enabled="true" id="21" needs-layout="false" visible="true"> <Tab bounds="0 0 100x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 12x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="82 0 99x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 35x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="163 0 99x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 35x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="244 0 99x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 35x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="16 14 496x23" enabled="true" id="0" needs-layout="false" visible="true" /> ``` _** Focus on root, tabs and label items_ Debug logs when switching to 1.25 scale: ``` [41531:41531:0808/155106.758722:ERROR:gtk_ui.cc(770)] OnGtkXftDpiChanged -------------------------------- [41531:41531:0808/155106.758789:ERROR:gtk_ui.cc(1059)] UpdateDeviceScaleFactor display_config changed. Updating and notifying. font_scale=1.25 [41531:41531:0808/155106.759036:ERROR:window_tree_host.cc(661)] OnDisplayMetricsChanged ==================== [41531:41531:0808/155106.759056:ERROR:window_tree_host.cc(669)] OnDisplayMetricsChanged ==================== will resize! [41531:41531:0808/155106.759094:ERROR:window_tree_host.cc(604)] OnHostResizedInPixels new_size_in_pixels=584x320 prev_scale=1 new_scale=1.25 bounds=0,0 584x320 [41531:41531:0808/155106.759130:ERROR:desktop_window_tree_host_platform.cc(898)] CalculateRootWindowBounds size_in_dip=468x256 [41531:41531:0808/155106.759143:ERROR:window_tree_host.cc(400)] UpdateRootWindowSize new_bounds=0,0 468x256 [41531:41531:0808/155106.759155:ERROR:window.cc(469)] SetBounds 2 BrowserFrame [41531:41531:0808/155106.759579:ERROR:compositor.cc(463)] SetScaleAndSize scale=1.25 size_in_pixel=584x320 [41531:41531:0808/155106.780150:ERROR:window.cc(469)] SetBounds 2 DesktopNativeWidgetAura - content window [41531:41531:0808/155106.786849:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [41531:41531:0808/155106.787052:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [41531:41531:0808/155106.787080:ERROR:window.cc(469)] SetBounds 2 RenderWidgetHostViewAura [41531:41531:0808/155106.787714:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [41531:41531:0808/155106.787732:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [41531:41531:0808/155106.787802:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [41531:41531:0808/155106.787813:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [41531:41531:0808/155106.787903:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [41531:41531:0808/155106.787916:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [41531:41531:0808/155106.789346:ERROR:window.cc(469)] SetBounds 2 NativeViewHostAuraClip [41531:41531:0808/155106.789365:ERROR:window.cc(469)] SetBounds 2 WebContentsViewAura [41531:41531:0808/155106.789624:ERROR:window_tree_host.cc(620)] OnHostResizedInPixels<<=================================== bounds=0,0 468x256 host_bounds=0,0 468x256 ``` ##### Font scale 1.25: **Buggy!** - Window bounds and/or scale seem to be out-of-sync at some level - Check buffer allocation in re-scale process (size) - Missing/erroneous protocol request? set_window_geometry? wp_viewporter?? ![[Pasted image 20240808155232.png]] | ![[Pasted image 20240808160728.png|500]] Views hierarchy: ``` <BrowserRootView bounds="0 0 468x256" enabled="true" id="0" needs-layout="false" visible="true"> <Label bounds="0 0 0x0" enabled="true" id="6" needs-layout="true" visible="false" /> <TabStrip bounds="28 0 227x41" enabled="true" id="21" needs-layout="false" visible="true"> <Tab bounds="0 0 71x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="20 12 7x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="53 0 70x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 6x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="105 0 70x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 6x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Tab bounds="157 0 70x41" enabled="true" id="20" needs-layout="false" visible="true"> <Label bounds="44 12 6x16" enabled="true" id="0" needs-layout="false" visible="true" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="0 0 0x0" enabled="true" id="0" needs-layout="false" visible="false" /> <Label bounds="16 14 380x23" enabled="true" id="0" needs-layout="false" visible="true" /> ``` #### Prototyping > CL: https://chromium-review.googlesource.com/5750512 ##### Building and testing ```bash export WAYLAND_DEBUG=1 autoninja -C out/linux chrome && \ chr_run --gtk-version=4 \ --enable-features=WaylandPerSurfaceScale,ui-debug-tools \ --vmodule="'wayland*=5'" \ --ozone-platform=wayland \ chromium.org /tmp/000* 2>&1 | \ grep -e 'ERROR\|VERBOSE\|^#[0-9]\+\|] $wl_surface\|wl_output\|wp_fractional$\|] -> wl_surface.*$attach\|damage\|commit\|set_input_region$\|] -> wp_viewport\|configure\| -> zwp_linux_buffer.*create' ``` ##### Status - [x] Hook into LinuxUi just like `X11ScreenOzone` + `XDisplayManager` - [x] Keep the font scale decoupled from display scale - Stored in `WaylandOutputManager`. Better place? - [x] Properly handle scale when processing the generated frame in `WaylandFrameManager` > `WaylandSurface`. - Factor the font scale out of the buffer scale when computing [WaylandSurface::GetWaylandScale](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_surface.cc;l=392-399;drc=5eb74477580abf22142fa6aa6b4de3e8312ac29e). ie: it's not a wayland "thing", it's just a window content scaling mechanism, which is not relevant to the Wayland compositor side. - Option 1: Access it directly from `WaylandOutputManager` - Implemented. Seems to work well. - Confirm with edcourtney@ if I'm missing something wrt config sync stuff. - Option 2: Have it as a new field in `PlatformWindowDelegate::State` ? - **Pending:** `pending_state_.buffer_scale_float` is sometimes used directly (instead of through `GetWaylandScale`). Refactor? - [x] Located input events (mouse, touch, etc) handling must be adapted - Similarly to what's done with `WaylandSurface::GetWaylandScale` - [x] Regular located events in [WaylandWindow](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=666;drc=8fffe4bbfe2f5011f716ec01083bb6dd33d047f9). Clients expect it in pixels, that's why `applied_state().window_scale` is included. - [x] Data drag controller events. Eg: [motion](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=862;drc=8fffe4bbfe2f5011f716ec01083bb6dd33d047f9). Clients expect it in DIP, thus only factor out the `font_scale_factor`. - [x] Window drag controller. Note: no changes needed, as actual dispatch is done by `WaylandWindow` - [ ] Fix popup windows - Eg: chrome's main menu. - [x] Double-scaling upon initialization. - [ ] Positioning (wrong) offset - [x] Window maximization and tiled states - [x] Bounds increase (sometimes) when font scale changes - [x] Fix GTK3 path ? - Currently broken. Fails to retrieve the font scale. - [x] Use GTK3's [Gtk.Settings:gtk-xft-dpi](https://docs.gtk.org/gtk3/property.Settings.gtk-xft-dpi.html) - [ ] Check since which GTK3 release it's supported - [ ] Support it in per-display scaling legacy path (optional) - For now supported only in per-surface path. - _Do we really want/need this?_ #### Risks ##### Rounding issues When trying to "preserve" bounds/size while re-scaling content, sometimes rounding issues may happen. How to address? Samples: - size=(old=1041x634, new=1042x635) scale=(old=1, new=1.25) ##### Design - WIP: [Ozone/Wayland: Large Text support - Google Docs](https://docs.google.com/document/d/16kwPm_S0dmxe4kSgKY0_PA7aFvkFcftx6AMzNqT8Nfg/edit#heading=h.3gwcxwf0g6qs) #### Viz-side flow Below are a few diagrams dissecting the main steps in new frames submission, focusing on the surface resize. ##### Root compositor frame submission (Viz entry point) ```mermaid sequenceDiagram participant RootFrameSink as RootCompositorFrameSinkImpl participant Display participant DamageTracker as DisplayDamageTracker participant DisplayScheduler participant FrameSinkSupport as CompositorFrameSinkSupport participant SurfaceManager participant Surface participant LayerContext as LayerContextImpl participant LayerTreeHostImpl RootFrameSink -) Display: SetLocalSurfaceId(local_surface_id, frame.device_scale_factor()) activate Display Note right of Display: Updates its `device_scale_factor_` class var. Used later when calling `renderer_->DrawFrame()` in DrawAndSwap()`. See the diagram below. deactivate Display RootFrameSink -) Display: Resize(frame.render_pass_list.back()->output_rect.size()) activate Display Display -) DamageTracker: damage_tracker_->DisplayResized() activate DamageTracker DamageTracker --) DisplayScheduler: OnDisplayDamaged(root_surface_id_) activate DisplayScheduler Note right of DisplayScheduler: Starts observing `BeginFrame` signals coming from the `BeginFrameSource` and starts its internal `base::DeadlineTimer`, which will later trigger the next `DrawAndSwap()` call, covered in the next diagram. deactivate DisplayScheduler deactivate DamageTracker deactivate Display RootFrameSink -) FrameSinkSupport: MaybeSubmitCompositorFrame(surface_id, frame) activate FrameSinkSupport FrameSinkSupport -) SurfaceManager: GetSurfaceForId or CreateSurface() FrameSinkSupport -) Surface: QueueFrame(frame, frame_index) Surface -) Surface: CommitFrame(frame, frame_index) Surface -) FrameSinkSupport: OnSurfaceCommitted(this) FrameSinkSupport -) FrameSinkSupport: UpdateNeedsBeginFramesInternal() deactivate FrameSinkSupport ``` ##### Scheduling a new frame after a resize ```mermaid sequenceDiagram autonumber participant DeadlineTimer as base::DeadlineTimer box Viz compositor thread participant DisplayScheduler participant Display participant DirectRenderer participant SkiaRenderer participant SkiaSurface as SkiaOutputSurfaceImpl end box Viz GPU thread participant SkiaSurfaceOnGpu as SkiaOutputSurfaceImplOnGpu participant OutputDevice as SkiaOutputDeviceBufferQueue participant OutputPresenter as OutputPresenterGL end box Ozone/Wayland participant GbmSurface as GbmSurfacelessWayland end DeadlineTimer --) DisplayScheduler: OnBeginFrameDeadline() activate DisplayScheduler DisplayScheduler -) Display: DrawAndSwap(params) activate Display note right of Display: `frame = surface_aggregator_.Aggregate(current_surface_id_, ...)` is called before DrawFrame(). Display -) DirectRenderer: DrawFrame(frame.render_pass_list, device_scale_factor_, current_surface_size) activate DirectRenderer DirectRenderer -) SkiaRenderer: Reshape(reshape_params) activate SkiaRenderer Note right of SkiaRenderer: Calls `viz::BufferQueue::Reshape()`. DirectRenderer -) SkiaSurface: Reshape(params) activate SkiaSurface Note right of SkiaSurface: Caches params and forwards them to the `SkiaOutputSurfaceImplOnGpu::Reshape` task scheduled to run on the gpu thread. SkiaSurface --) SkiaSurfaceOnGpu: Reshape(params) activate SkiaSurfaceOnGpu SkiaSurfaceOnGpu -) OutputDevice: Reshape(params) OutputDevice -) OutputPresenter: Reshape(params) OutputPresenter -) GbmSurface: Resize(params) Note over GbmSurface: Sets its `surface_scale_factor_`. Later used when `CommitOverlays()` is called. deactivate SkiaSurfaceOnGpu deactivate SkiaSurface deactivate SkiaRenderer DirectRenderer -) SkiaRenderer: FinishDrawingFrame() activate SkiaRenderer Note right of SkiaRenderer: Add an overlay candidate for frame's output_surface_plane SkiaRenderer -) SkiaRenderer: ScheduleOverlays() activate SkiaRenderer SkiaRenderer -) SkiaSurface: ScheduleOverlays(current_frame()->overlay_list, sync_tokens) Note right of SkiaSurface: Enqueues a new GPU task for `SkiaOutputSurfaceImplOnGpu::ScheduleOverlays`. SkiaSurface --) SkiaSurfaceOnGpu: ScheduleOverlays(overlays) deactivate SkiaRenderer deactivate SkiaRenderer deactivate DirectRenderer Display -) SkiaRenderer: SwapBuffers(swap_frame_data) activate SkiaRenderer SkiaRenderer -) SkiaSurface: SwapBuffers(output_frame) Note right of SkiaSurface: Enqueues a new GPU task for `SkiaOutputSurfaceImplOnGpu::SwapBuffers`. SkiaSurface --) SkiaSurfaceOnGpu: SwapBuffers(frame) Note right of SkiaSurfaceOnGpu: Further details in the diagram below. deactivate SkiaRenderer deactivate Display deactivate DisplayScheduler ``` Remarks: - From [crrev.com/2684674](https://chromium-review.googlesource.com/c/chromium/src/+/2684674) on, some padding was added to buffers allocated for optimizing resizes. Core logic is in this [function](https://source.chromium.org/chromium/chromium/src/+/main:components/viz/service/display/direct_renderer.cc;l=1124;drc=a8089279ed05c89d4fa360414bc79618faeacbdd). ##### Swapping buffers (GPU thread) ```mermaid sequenceDiagram autonumber box Viz GPU thread participant GpuScheduler as gpu::SchedulerDfs participant SkiaSurfaceOnGpu as SkiaOutputSurfaceImplOnGpu participant OutputDevice as SkiaOutputDeviceBufferQueue participant OutputPresenter as OutputPresenterGL end box Ozone/Wayland (GPU Process) participant GbmSurface as GbmSurfacelessWayland participant GbmPixmap as GbmPixmapWayland participant BufferManagerGpu as WaylandBufferManagerGpu end box Ozone/Wayland (Browser Process) participant BufferManagerHost as WaylandBufferManagerHost end GpuScheduler --) SkiaSurfaceOnGpu: SwapBuffers(frame) SkiaSurfaceOnGpu -) OutputDevice: Submit(sync_cpu, PostSubmit, frame) OutputDevice --) SkiaSurfaceOnGpu: PostSubmit(frame) activate SkiaSurfaceOnGpu SkiaSurfaceOnGpu -) OutputDevice: ScheduleOverlays(overlays) activate OutputDevice loop For each overlay in overlays OutputDevice -) OutputPresenter: ScheduleOverlayPlane(overlay, access, acquire_fence) OutputPresenter -) GbmSurface: ScheduleOverlayPlane(image, gpu_fence, data) activate GbmSurface GbmSurface -) GbmPixmap: image->ScheduleOverlayPlane(widget_, ...) activate GbmPixmap GbmPixmap -) BufferManagerGpu: buffer_manager_->CreateDmabufBasedBuffer(fd, size_, ...) activate BufferManagerGpu BufferManagerGpu --) BufferManagerHost: remote_host_->CreateDmabufBasedBuffer(fd, size_, ...) deactivate BufferManagerGpu deactivate GbmPixmap note right of GbmSurface: After ensuring a buffer is allocated, appends a new OverlayConfig to `unsubmitted_frames_.back().configs`. `surface_scale_factor_` is used as `WaylandOverlayConfig.surface_scale_factor`. deactivate GbmSurface end deactivate OutputDevice SkiaSurfaceOnGpu -) OutputDevice: SetViewportSize(frame->size) Note right of OutputDevice: Sets a class var that seems unused atm. SkiaSurfaceOnGpu -) OutputDevice: Present(sub_buffer_rect, presented_callback_, frame) activate OutputDevice OutputDevice -) OutputPresenter: Present(DoFinishSwapBuffers, feedback_callback, frame.data) activate OutputPresenter OutputPresenter -) GbmSurface: Present(completion_cb, feedback_cb, data) activate GbmSurface Note right of GbmSurface: Create and append a new item to `unsubmitted_frames_` GbmSurface -) GbmSurface: MaybeSubmitFrames() activate GbmSurface loop For each frame in `unsubmitted_frames_` where frame.ready == true GbmSurface -) BufferManagerGpu: CommitOverlays(widget_, frame->frame_id, frame->data, frame->configs) BufferManagerGpu --) BufferManagerHost: remote_host_->CommitOverlays(widget, frame_id, data, configs) end deactivate GbmSurface Note right of GbmSurface: If linux_explicit_sync is not supported, frame submission will wait for fences. deactivate GbmSurface deactivate OutputPresenter deactivate OutputDevice deactivate SkiaSurfaceOnGpu ``` ### Fonts initialization ``` [107748:107748:0805/193717.190045:ERROR:gtk_ui.cc(354)] InitializeFontSettings ++++++ reset font render params. size_pixels=21 #0 0x78bb2cf84832 base::debug::CollectStackTrace() [../../base/debug/stack_trace_posix.cc:1044:7] #1 0x78bb2cf60dca base::debug::StackTrace::StackTrace() [../../base/debug/stack_trace.cc:245:20] #2 0x78bb1347db07 gtk::GtkUi::InitializeFontSettings() [../../ui/gtk/gtk_ui.cc:357:17] #3 0x78bb23144171 ui::LinuxUi::GetDefaultFontDescription() #4 0x78bb2a3c26fb gfx::PlatformFontSkia::EnsuresDefaultFontIsInitialized() [../../ui/gfx/platform_font_skia.cc:174:43] #5 0x78bb2a3c2619 gfx::PlatformFontSkia::PlatformFontSkia() [../../ui/gfx/platform_font_skia.cc:101:3] #6 0x78bb2a3c444b gfx::PlatformFont::CreateDefault() [../../ui/gfx/platform_font_skia.cc:468:14] #7 0x78bb2a39249e gfx::Font::Font() [../../ui/gfx/font.cc:22:31] #8 0x78bb2a39390b gfx::FontList::GetDefaultImpl() [../../ui/gfx/font_list.cc:232:30] #9 0x78bb2a39386e gfx::FontList::FontList() [../../ui/gfx/font_list.cc:109:30] #10 0x78bb2b1ba13e ui::ResourceBundle::GetFontListForDetails() [../../ui/base/resource/resource_bundle.cc:864:37] #11 0x78bb22fac529 views::TypographyProvider::GetFont() [../../ui/views/style/typography_provider.cc:30:50] #12 0x78bb22eb93a9 views::LabelButton::LabelButton() [../../ui/views/controls/button/label_button.cc:58:37] #13 0x61bd47bb0a5f TabStripControlButton::TabStripControlButton() [../../chrome/browser/ui/views/tabs/tab_strip_control_button.cc:88:7] #14 0x61bd47bb0983 TabStripControlButton::TabStripControlButton() [../../chrome/browser/ui/views/tabs/tab_strip_control_button.cc:65:7] #15 0x61bd47b5f7f6 TabSearchButton::TabSearchButton() [../../chrome/browser/ui/views/tabs/tab_search_button.cc:30:7] #16 0x61bd47b5d282 TabSearchContainer::TabSearchContainer() [../../third_party/libc++/src/include/__memory/unique_ptr.h:634:30] #17 0x61bd478e0715 TabStripRegionView::TabStripRegionView() [../../third_party/libc++/src/include/__memory/unique_ptr.h:634:30] #18 0x61bd47837f21 BrowserView::BrowserView() [../../third_party/libc++/src/include/__memory/unique_ptr.h:634:30] #19 0x61bd478bcef8 BrowserWindow::CreateBrowserWindow() [../../chrome/browser/ui/views/frame/browser_window_factory.cc:61:14] #20 0x61bd47378618 Browser::Browser() [../../chrome/browser/ui/browser.cc:320:10] #21 0x61bd473778c7 Browser::Create() [../../chrome/browser/ui/browser.cc:531:14] #22 0x61bd47418725 StartupBrowserCreatorImpl::OpenTabsInBrowser() [../../chrome/browser/ui/startup/startup_browser_creator_impl.cc:249:15] #23 0x61bd47419af4 StartupBrowserCreatorImpl::RestoreOrCreateBrowser() [../../chrome/browser/ui/startup/startup_browser_creator_impl.cc:632:13] #24 0x61bd47417ebb StartupBrowserCreatorImpl::DetermineURLsAndLaunch() [../../chrome/browser/ui/startup/startup_browser_creator_impl.cc:449:22] #25 0x61bd4741739f StartupBrowserCreatorImpl::Launch() [../../chrome/browser/ui/startup/startup_browser_creator_impl.cc:178:7] #26 0x61bd47412ff6 StartupBrowserCreator::LaunchBrowser() [../../chrome/browser/ui/startup/startup_browser_creator.cc:723:9] #27 0x61bd47414221 StartupBrowserCreator::ProcessLastOpenedProfiles() [../../chrome/browser/ui/startup/startup_browser_creator.cc:1408:5] #28 0x61bd47413c93 StartupBrowserCreator::LaunchBrowserForLastProfiles() [../../chrome/browser/ui/startup/startup_browser_creator.cc:822:3] #29 0x61bd47412985 StartupBrowserCreator::ProcessCmdLineImpl() [../../chrome/browser/ui/startup/startup_browser_creator.cc:1359:3] #30 0x61bd4741172e StartupBrowserCreator::Start() [../../chrome/browser/ui/startup/startup_browser_creator.cc:674:10] #31 0x61bd456c2afd ChromeBrowserMainParts::PreMainMessageLoopRunImpl() [../../chrome/browser/chrome_browser_main.cc:1799:25] #32 0x61bd456c1f7a ChromeBrowserMainParts::PreMainMessageLoopRun() [../../chrome/browser/chrome_browser_main.cc:1244:18] #33 0x78bb28f50cb7 content::BrowserMainLoop::PreMainMessageLoopRun() [../../content/browser/browser_main_loop.cc:1007:28] #34 0x78bb28e54430 base::OnceCallback<>::Run() [../../base/functional/callback.h:156:12] #35 0x78bb29897b59 content::StartupTaskRunner::RunAllTasksNow() [../../content/browser/startup_task_runner.cc:42:29] #36 0x78bb28f50784 content::BrowserMainLoop::CreateStartupTasks() [../../content/browser/browser_main_loop.cc:910:25] #37 0x78bb28f53f81 content::BrowserMainRunnerImpl::Initialize() [../../content/browser/browser_main_runner_impl.cc:137:15] #38 0x78bb28f4dd4d content::BrowserMain() [../../content/browser/browser_main.cc:30:32] #39 0x78bb29f6338b content::RunBrowserProcessMain() [../../content/app/content_main_runner_impl.cc:735:10] #40 0x78bb29f65f36 content::ContentMainRunnerImpl::RunBrowser() [../../content/app/content_main_runner_impl.cc:1321:10] #41 0x78bb29f655d8 content::ContentMainRunnerImpl::Run() [../../content/app/content_main_runner_impl.cc:1173:12] #42 0x78bb29f61bcf content::RunContentProcess() [../../content/app/content_main.cc:333:36] #43 0x78bb29f61e32 content::ContentMain() [../../content/app/content_main.cc:346:10] #44 0x61bd4411aaa0 ChromeMain [../../chrome/app/chrome_main.cc:230:12] #45 0x78bb16591e08 (/usr/lib/libc.so.6+0x25e07) #46 0x78bb16591ecc __libc_start_main #47 0x61bd4411a6ca _start ``` - `ResourceBundle` has an internal fonts list (metadata) cache - [ResourceBundle::font\_cache\_](https://source.chromium.org/chromium/chromium/src/+/main:ui/base/resource/resource_bundle.h;l=591;drc=41374c974d98f8cf67134f9ddb8d96d398154dfe) - It also has a [ReloadFonts()](https://source.chromium.org/chromium/chromium/src/+/main:ui/base/resource/resource_bundle.cc;l=927-931;drc=41374c974d98f8cf67134f9ddb8d96d398154dfe) function, which clears the cache - Single sample usage [here](https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/ash/base/locale_util.cc;l=113;drc=41374c974d98f8cf67134f9ddb8d96d398154dfe) - ### Omnibox text view - [OmniboxTextView::CreateRenderText](https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/ui/views/omnibox/omnibox_text_view.cc;l=342;drc=41374c974d98f8cf67134f9ddb8d96d398154dfe) - ELIDE_TAIL is used - Resizeable? - ... ### Concepts Some relevant concepts are listed in [[Wayland notes#General graphics concepts|Chromium graphics concepts]] section. --- ## Configure sync - New frames (ie: buffers + overlays) might need to be produced either (1) in response to surface configure events coming from the wayland compositor, or (2) arbitrarily triggered by client. - The frames creation happens asynchronously, and in Chromium they're usually produced by the Gpu/Viz process. - Once new frames are produced and sent to the compositor, done by the browser process, a set of requests must also be sent at some point to let the compositor know what states were applied for that frame, eg: window geometry, etc. Besides that, if that frame was produced in response to a configure sequence, an `ack_configure` must also be sent. ### Implementation - [PlatformWindowDelegate\:\:State](https://source.chromium.org/chromium/chromium/src/+/main:ui/platform_window/platform_window_delegate.h;l=113;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35) holds the relevant metadata of the state change requests, for both cases described above. - [WaylandWindow::RequestState](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=1329;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35) is the entry point to request a new state change, which may (or may not) produce a new frame. - The requested state is stored window's [in\_flight\_requests_](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.h;l=711;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35) - It then calls into [WaylandWindow::MaybeApplyLatestStateRequest](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=1520;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35), which in turn calls [PlatformWindowDelegate::OnStateUpdate](https://source.chromium.org/chromium/chromium/src/+/main:ui/aura/window_tree_host_platform.cc;l=354;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35) and marks it as [applied](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=1549;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35); - [WTHPlatform::OnStateUpdate](https://source.chromium.org/chromium/chromium/src/+/main:ui/aura/window_tree_host_platform.cc;l=398-407;drc=63635fff4bce239256a42d5f1a28fa3d0934bf35) is responsible for allocating and returning a "sequence id" which will be used later to match a new frame corresponding to that state request and ack it if needed (ie: when there is a serial number associated); - [WaylandWindow::ProcessSequencePoint](https://source.chromium.org/chromium/chromium/src/+/main:ui/ozone/platform/wayland/host/wayland_window.cc;l=1436;drc=5203366d4cf174b9331ee33563e0520f10b60828) is called when a new frame, corresponding to a state change request, is received (from Viz). - It updates `latched_state_` - Issues `xdg_surface.set_window_geometry` in case of bounds change - and `ack_configure` if the state changes are associated with a configure sequence ### Producing new frames [[Support platform-controlled window-driven scaling in Chromium#Appendix#Producing new frames|Per-window scaling]] doc contains a deep analysis on how new frames are triggered when scale factor changes.