Technology preview announcement for agl-compositor


Marius Vlad
 

Hi,

As some of you might be aware Collabora has been working, for some time,
on a new compositor/display infrastructure which we believe is ready for
a technology preview for those who want to give it a try.

With this announcement we would like to write a few words about what has
changed, how to set it up, and what are the next potential action items
to look into.

Note this is on-going work, and might suffer some intermediary breakage.
Some of the details are still a bit fuzzy/in discussion and might suffer
additional changes.

# Terminology, current architecture and changes

Wayland, as most of you know, is only a protocol, with some primitives
to allow marshalling data between a client and a server and with some
higher protocol specification on top of those primitives
defined with the help of a XML file. The protocol itself used to (and
still does, but it has since been deprecated) have window management
functionality (wl_shell) but as those requirements have been largely
revised it was decided to migrate those to its own protocol extension
called xdg-shell.

Weston is a server that implements the Wayland protocol. It is focused
on correctness and efficient HW usage. Weston is built on top of
libweston which has a front-facing API and an internal API to abstract
the HW and allow the compositor developer focus on providing the
required functionality as they deems necessary. Another compositor built
on top of libweston is agl-compositor, which is the main subject of this
email. Besides libweston, weston architecture was designed to allow
plug-ins to be loaded at run-time which can implement even more
functionality. But more on that later, a few paragraphs below.

xdg-shell is currently de facto protocol for handling all window
management related functionality. In order to avoid polluting the
wayland protocol namespace, the XDG extension, together with some other
useful protocols, like the linux-dmabuf one, are all developed under the
wayland-protocols umbrella. The whole purpose of wayland-protocols is to
enhance the Wayland protocol with new functionality and bring new
extensions entirely. Compositors are free to implement, modify, enhance,
and add new extensions to wayland-protocols but they need to do so in
consensus.

Besides the core wayland protocol and extended functionality from
wayland-protocols, a compositor can provide additional protocol
extension specifications (obviously specific to that compositor).
Another window management extension instead of xdg-shell, that abides to
the GENIVI specification (Infotainment), is the ivi-shell:
a Weston module that implements ivi-application protocol specification.
Together with this ivi-shell, further additional functionality has also
been brought in with wayland-ivi-extension (which allows, among other
things, to specify the position of windows).

The ivi-shell plugin has been designed to allow loading of additional
modules, called controllers, which each vendor can adapt and modify on
their own. Weston provides hmi-controller (as an example), and
wayland-ivi-extension provides ivi-controller.

Graphical widget toolkits like Qt will abstract the underlying window
management in use, and can at run-time, switch between ivi-shell and
xdg-shell. In the same manner, the windowmanager currently found in AGL
is responsible for abstracting the ivi-shell interaction and it used for
managing and controlling the Layers, Screens and Surfaces from ivi-shell.

Last in our architecture is the graphical user interface called
homescreen and is implemented with Qt.
It spawns applications and switches to/activates/displays them. It does
that by using a library (libhomescreen) which is another abstraction
built on top of a binder, called agl-service-homescreen.
The binder talks with afm-system-daemon to start the application (if not
already started) prior to switching/activating it by using
windowmanager/libwindowmanager and the service with the same.
The binder can also be used to switch/activate other applications and it
is also in use by the launcher application.

With the introduction of agl-compositor we aim to simplify the design
and trim down this chain of abstractions while retaining the ability to
modify and adapt for each vendor's needs and requirements.

The approach we have taken is to create a new, simpler, but powerful
compositor and to use xdg-shell instead of relying further on ivi-shell
and its convoluted design. The compositor is built on top of libweston,
and has been designed to contain only the bare minimum to bring up the
display and the main event loop. As all the heavy lifting is done inside
libweston, vendors are encouraged to study, adapt, and modify the
compositor as they see fit.

The xdg-shell server side implementation is hidden behind
libweston-desktop library which agl-compositor is a user of, in the same
way weston is also user of it (libweston-desktop) and it's using it for
implementing its desktop-shell. This allows regular xdg-shell clients be
managed by the compositor without further interaction and in a
transparent manner.

Circling back to the Qt-based UI, homescreen will now contain
functionality to instruct the compositor to switch between regular XDG
toplevel surfaces by using a private protocol called agl-shell. The
protocol implementation explicitly assigns roles to different parts of
the output: panels (top, left, right, bottom), the background, and
desktop (for regular xdg-shell clients). We've adapted homescreen as to
use QPA (Qt Platform Abstraction) to bind to the agl-shell protocol
interface as to integrate the current functionality we already have in
homescreen: a top panel that contains a list of applications and a
bottom panel that contains volume UI. Each of these represents a surface
and it is a 1-to-1 assignment from a QML Window type, including the
background.

In terms of desktop role surfaces (or regular XDG clients) the
mediaplayer, the launcher, hvac have suffered some slight
modification, mostly QML hard-coded width/heigth or the removal of
wrappers on top Qt Core GUI. The AGL image is built and deployed in such
a manner that the shell is automatically picked from an exported
environment variable so the recommendation is that users should just use
the regular Qt API when creating new applications and let Qt itself
handle it.

Do note, that while these application "conversions" should display
correctly when using agl-compositor, some of the "regular" functionality
might be missing as the focus was towards bring-up of the display part
rather than AFM integration.

# Further actionable items

As some of the functionality disappeared because
libhomescreen/libwindowmanager and the homescreen service were no longer
necessary, homescreen now acts as UI/window manager/desktop manager and
controls, together with the compositor, launching new applications and
switching to them. As explained above, the ability to launch
application is currently hidden from homescreen by using libhomescreen
and the agl-service-homescreen.

For the moment, homescreen contains some potential duplicated code (for
launching/starting applications) which should probably reside in a
generic library which others (like the launcher for instance)
can use in a programmatic fashion (and not only for launching
applications). Adaptation of the launcher code also has this same
duplication part when retrieving the list of runable applications.

The activation of top level XDG surfaces is currently done from within
homescreen, homescreen being just an example on how that can be
achieved. Vendors (and users alike) can modify, just like the
compositor, and adapt it as they seem fit.

The need to activate/display other applications (but without homescreen
involvement) lead us to add an additional protocol extension, which we
called agl-shell-desktop. Regular applications can bind to it and
receive application_id advertisements whenever new regular XDG
application have been mapped in the compositor but also to switch
to/activate them.

This should allow to preserve the same functionality we currently have
in AGL, respectively in agl-service-homescreen. As it stands now there
is no code to handle it in the compositor, but will probably make its
way to the compositor quite fast once next is back-merged into master.

Some of those changes reside in a sandbox repository @
https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=shortlog;h=refs/heads/sandbox/mvlad/next-wip

To conclude, we split the launching/spawning part of applications, which
we believe should be handled independently by using a regular library to
talk with afm-system-daemon and the activation part. The
activation/switching part, can be handled from within agl-shell, while
for regular XDG application should be done using agl-shell-desktop.

We could probably make this even clearer by having agl-shell just
deal with surface roles and agl-shell-desktop for handling switching
of applications but for the time being they will be independent of each
other.

# How to test/use for the time being

## Building

In order to test the compositor and with it, the Qt UI (homescreen), one
must enable the 'agl-compositor' DISTRO_FEATURE. This requires a
regeneration of the conf/local.conf file so when sourcing the aglsetup
script you must pass the -f environment flag:

$ source meta-agl/scripts/aglsetup.sh -f -b $MACHINETYPE agl-demo
agl-hmi-framework agl-compositor

where MACHINETYPE is the machine you're building for.

This will build the compositor from master branch and will switch
homescreen and apps like mediaplayer/hvac/mediaplayer/launcher to a
sandbox repository which contains the converted/adapted applications.

Obviously you will need to use bitbake and generate a workable image. If
there's no intermediary breakage you should see the regular homescreen
starting up upon booting the image.

Both homescreen and the regular applications are using
sandbox/mvlad/agl-composositor as the integration branch for usage with
agl-compositor, and can be pulled individually as well from the
following links:

- homescreen:
https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/homescreen.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor
- hvac:
https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/hvac.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor
- mediaplayer:
https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/mediaplayer.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor
- launcher
https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/launcher.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor

Note that these require quite a few additional dependencies so the
easiest is to to use yocto directly and qemu as MACHINETYPE.

## Start/stop agl-compositor

The DISTRO_FEATURE 'agl-compositor' will replace the path of the
compositor (currently weston) in the weston unit systemd file
to that of agl-compositor, so you should start/stop the compositor using
the same unit systemd file (weston).

## Switching application by by default

Applications that are not started by homescreen, will not be switched
to/activated by default, so we've added an option to have this happening
by default, but you need to create a agl-compositor.ini file edit add
the the following:

[shell]
activate-by-default=true

Currently, you'll need to add this agl-compositor.ini file argument
list in the unit systemd for the weston systemd service file and reload
it in order for the compositor to be aware of it. Take a look at the log
file to see if the compositor loads the config file accordingly.

With this activated, you should be able to start applications, as
agl-driver user, using afm-util and they should be activated/switched to
upon starting.

Note that the compositor is aware of other sections like '[output]'
which are required in case you need to change the orientation of the
output (just like weston).

## Black surface and delayed presentation

As long as there's no client binded to the agl-shell interface or if the
client unbinded/has been closed, a black surface will be displayed. This
is intentional and is a perfect example for instance if vendors
would like to modify this behaviour or not.

Also, the agl-shell protocol will delay presentation until the client
shell sends a 'ready' request.

For more details about the protocol I'd suggest taking a look at:

https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell.xml;hb=refs/heads/master

## Local testing

agl-compositor can be built and install locally, with the following
being needed:

- wayland-protocols
- weston (depending on the branch 6 or 7)

while the compositor itself can be found at:
https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-compositor

Note that multiple backends exists, like x11 and wayland (nested), but
also a native DRM backend, so you should be able to test it directly
under different set-ups.

For the adventurous, but also for further details regarding client
examples:

- qtshell (homescreen like simple example)
https://gitlab.collabora.com/mvlad/qtshell
- to-activate (mimics a regular XDG application)
https://gitlab.collabora.com/mvlad/to-activate
- activator (mimics a regular XDG application but uses another
extension, agl-shell-dekstop to activate other
apps like ``to-activate'' above app)
https://gitlab.collabora.com/mvlad/activator

These client applications do require Qt and some additional AGL
packages: libqtappfw and afb-helpers-qt which can be built locally or
installed from other parts (depends on your package
manager/distribution) but require less dependencies that homescreen or
other regular applications. The repositories contain a yocto recipe but
they have to be adapted a bit to make them work correctly.

# In the future

For now the focus was Qt and Qt based UI, but we'll start looking soon
into WAM (WebApplicationManager) to determine what would be needed to
make it work with agl-compositor and into the policy manager from the
windowmanager service and see how that can be adapted for using it
with/within the compositor.

The Qt application would also need some up-do-date changes as they seem
a bit outdated at the moment (this was done around Christmas time) and
are in need of further validation of functionality.

--
Marius Vlad

Join agl-dev-community@lists.automotivelinux.org to automatically receive all group messages.