Topics

Tips for debugging GPU acceleration.


Alex Bennée
 

Hi Gerd,

I'm currently trying to get accelerated graphics working on an AArch64
AGL distribution (both in KVM and TCG builds). As far as I can tell the
device is being correctly created and I do get a working DRI based
display. However the Weston based compositor falls back to the softpipe
rendering:

Date: 2020-10-22 UTC
[19:37:22.296] Using config file '/etc/xdg/weston/weston.ini'.
[19:37:22.297] Output repaint window is 7 ms maximum.
[19:37:22.298] Loading module '/usr/lib/libweston-8/drm-backend.so'
[19:37:22.318] initializing drm backend
[19:37:22.323] logind: not running in a systemd session
[19:37:22.323] logind: cannot setup systemd-logind helper (-61), using legacy fallback
[19:37:22.370] using /dev/dri/card0
[19:37:22.370] DRM: supports universal planes
[19:37:22.370] DRM: supports atomic modesetting
[19:37:22.370] DRM: supports picture aspect ratio
[19:37:22.373] Loading module '/usr/lib/libweston-8/gl-renderer.so'
[19:37:22.540] EGL client extensions: EGL_EXT_client_extensions
EGL_EXT_device_base EGL_EXT_device_enumeration
EGL_EXT_device_query EGL_EXT_platform_base
EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug
EGL_EXT_platform_wayland EGL_MESA_platform_gbm
EGL_MESA_platform_surfaceless EGL_EXT_platform_device
[19:37:22.542] EGL version: 1.4
[19:37:22.542] EGL vendor: Mesa Project
[19:37:22.542] EGL client APIs: OpenGL OpenGL_ES
[19:37:22.542] EGL extensions: EGL_ANDROID_blob_cache EGL_EXT_buffer_age
EGL_EXT_image_dma_buf_import EGL_KHR_cl_event2
EGL_KHR_config_attribs EGL_KHR_create_context
EGL_KHR_create_context_no_error EGL_KHR_fence_sync
EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace
EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image
EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image
EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap
EGL_KHR_no_config_context EGL_KHR_reusable_sync
EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float
EGL_KHR_wait_sync EGL_MESA_configless_context
EGL_MESA_image_dma_buf_export EGL_MESA_query_driver
[19:37:22.542] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension
[19:37:22.542] EGL_KHR_surfaceless_context available
[19:37:22.819] GL version: OpenGL ES 3.1 Mesa 20.0.2
[19:37:22.819] GLSL version: OpenGL ES GLSL ES 3.10
[19:37:22.819] GL vendor: VMware, Inc.
[19:37:22.819] GL renderer: softpipe
[19:37:22.819] GL extensions: GL_EXT_blend_minmax GL_EXT_multi_draw_arrays

I know the kernel has the right bits enabled (VIRTIO_DRM_GPU) but the
kernel seems to think the host needs something else to support it:

[ 2.743035] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)
[ 2.746441] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)
[ 2.778843] random: fast init done
[ 2.780024] random: crng init done
[ 2.781054] bochs-drm 0000:00:02.0: remove_conflicting_pci_framebuffers: bar 0: 0x10000000 -> 0x10ffffff
[ 2.781597] bochs-drm 0000:00:02.0: remove_conflicting_pci_framebuffers: bar 2: 0x11014000 -> 0x11014fff
[ 2.786609] bochs-drm 0000:00:02.0: enabling device (0000 -> 0002)
[ 2.789259] [drm] Found bochs VGA, ID 0xb0c5.
[ 2.789639] [drm] Framebuffer size 16384 kB @ 0x10000000, mmio @ 0x11014000.
[ 2.791527] [TTM] Zone kernel: Available graphics memory: 2015562 KiB
[ 2.792300] [TTM] Initializing pool allocator
[ 2.793275] [TTM] Initializing DMA pool allocator
[ 2.801629] [drm] Found EDID data blob.
[ 2.809735] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:02.0 on minor 0
[ 2.903431] Console: switching to colour frame buffer device 128x48
[ 2.928141] bochs-drm 0000:00:02.0: fb0: bochs-drmdrmfb frame buffer device
[ 2.929844] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0
[ 2.930300] [drm] virgl 3d acceleration not supported by host
[ 2.930484] [drm] EDID support available.
[ 2.931193] [drm] number of scanouts: 1
[ 2.931318] [drm] number of cap sets: 0
[ 2.935329] [drm] Initialized virtio_gpu 0.1.0 0 for virtio4 on minor 1
[ 3.058526] virtio_gpu virtio4: fb1: virtio_gpudrmfb frame buffer device

I'm invoking QEMU on both systems with:

-device virtio-gpu-pci,virgl=true

The full TCG command line is:

./aarch64-softmmu/qemu-system-aarch64 -cpu cortex-a57 \
-machine type=virt,virtualization=on,gic-version=3 \
-serial mon:stdio \
-netdev user,id=unet,hostfwd=tcp::2222-:22 \
-device virtio-net-device,netdev=unet,id=virt-net \
-drive id=disk0,file=$HOME/images/agl/agl-demo-platform-crosssdk-qemuarm64.ext4,if=none,format=raw \
-device virtio-blk-device,drive=disk0 \
-soundhw hda \
-device qemu-xhci \
-device usb-tablet \
-device usb-kbd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-device virtio-serial-device \
-chardev null,id=virtcon \
-device virtconsole,chardev=virtcon \
-m 4096 \
-kernel ~/images/agl/Image \
-append "console=ttyAMA0 root=/dev/vda" \
-display gtk,show-cursor=on \
-smp 4 \
-device virtio-gpu-pci,virgl=true

Any tips on how to track down where things are going wrong?

--
Alex Bennée


Alex Bennée
 

Alex Bennée <alex.bennee@...> writes:

(fix the qemu-devel address)

Hi Gerd,

I'm currently trying to get accelerated graphics working on an AArch64
AGL distribution (both in KVM and TCG builds). As far as I can tell the
device is being correctly created and I do get a working DRI based
display. However the Weston based compositor falls back to the softpipe
rendering:

Date: 2020-10-22 UTC
[19:37:22.296] Using config file '/etc/xdg/weston/weston.ini'.
[19:37:22.297] Output repaint window is 7 ms maximum.
[19:37:22.298] Loading module '/usr/lib/libweston-8/drm-backend.so'
[19:37:22.318] initializing drm backend
[19:37:22.323] logind: not running in a systemd session
[19:37:22.323] logind: cannot setup systemd-logind helper (-61), using legacy fallback
[19:37:22.370] using /dev/dri/card0
[19:37:22.370] DRM: supports universal planes
[19:37:22.370] DRM: supports atomic modesetting
[19:37:22.370] DRM: supports picture aspect ratio
[19:37:22.373] Loading module '/usr/lib/libweston-8/gl-renderer.so'
[19:37:22.540] EGL client extensions: EGL_EXT_client_extensions
EGL_EXT_device_base EGL_EXT_device_enumeration
EGL_EXT_device_query EGL_EXT_platform_base
EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug
EGL_EXT_platform_wayland EGL_MESA_platform_gbm
EGL_MESA_platform_surfaceless EGL_EXT_platform_device
[19:37:22.542] EGL version: 1.4
[19:37:22.542] EGL vendor: Mesa Project
[19:37:22.542] EGL client APIs: OpenGL OpenGL_ES
[19:37:22.542] EGL extensions: EGL_ANDROID_blob_cache EGL_EXT_buffer_age
EGL_EXT_image_dma_buf_import EGL_KHR_cl_event2
EGL_KHR_config_attribs EGL_KHR_create_context
EGL_KHR_create_context_no_error EGL_KHR_fence_sync
EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace
EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image
EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image
EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap
EGL_KHR_no_config_context EGL_KHR_reusable_sync
EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float
EGL_KHR_wait_sync EGL_MESA_configless_context
EGL_MESA_image_dma_buf_export EGL_MESA_query_driver
[19:37:22.542] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension
[19:37:22.542] EGL_KHR_surfaceless_context available
[19:37:22.819] GL version: OpenGL ES 3.1 Mesa 20.0.2
[19:37:22.819] GLSL version: OpenGL ES GLSL ES 3.10
[19:37:22.819] GL vendor: VMware, Inc.
[19:37:22.819] GL renderer: softpipe
[19:37:22.819] GL extensions: GL_EXT_blend_minmax GL_EXT_multi_draw_arrays

I know the kernel has the right bits enabled (VIRTIO_DRM_GPU) but the
kernel seems to think the host needs something else to support it:

[ 2.743035] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)
[ 2.746441] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)
[ 2.778843] random: fast init done
[ 2.780024] random: crng init done
[ 2.781054] bochs-drm 0000:00:02.0: remove_conflicting_pci_framebuffers: bar 0: 0x10000000 -> 0x10ffffff
[ 2.781597] bochs-drm 0000:00:02.0: remove_conflicting_pci_framebuffers: bar 2: 0x11014000 -> 0x11014fff
[ 2.786609] bochs-drm 0000:00:02.0: enabling device (0000 -> 0002)
[ 2.789259] [drm] Found bochs VGA, ID 0xb0c5.
[ 2.789639] [drm] Framebuffer size 16384 kB @ 0x10000000, mmio @ 0x11014000.
[ 2.791527] [TTM] Zone kernel: Available graphics memory: 2015562 KiB
[ 2.792300] [TTM] Initializing pool allocator
[ 2.793275] [TTM] Initializing DMA pool allocator
[ 2.801629] [drm] Found EDID data blob.
[ 2.809735] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:02.0 on minor 0
[ 2.903431] Console: switching to colour frame buffer device 128x48
[ 2.928141] bochs-drm 0000:00:02.0: fb0: bochs-drmdrmfb frame buffer device
[ 2.929844] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0
[ 2.930300] [drm] virgl 3d acceleration not supported by host
[ 2.930484] [drm] EDID support available.
[ 2.931193] [drm] number of scanouts: 1
[ 2.931318] [drm] number of cap sets: 0
[ 2.935329] [drm] Initialized virtio_gpu 0.1.0 0 for virtio4 on minor 1
[ 3.058526] virtio_gpu virtio4: fb1: virtio_gpudrmfb frame buffer device

I'm invoking QEMU on both systems with:

-device virtio-gpu-pci,virgl=true

The full TCG command line is:

./aarch64-softmmu/qemu-system-aarch64 -cpu cortex-a57 \
-machine type=virt,virtualization=on,gic-version=3 \
-serial mon:stdio \
-netdev user,id=unet,hostfwd=tcp::2222-:22 \
-device virtio-net-device,netdev=unet,id=virt-net \
-drive id=disk0,file=$HOME/images/agl/agl-demo-platform-crosssdk-qemuarm64.ext4,if=none,format=raw \
-device virtio-blk-device,drive=disk0 \
-soundhw hda \
-device qemu-xhci \
-device usb-tablet \
-device usb-kbd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-device virtio-serial-device \
-chardev null,id=virtcon \
-device virtconsole,chardev=virtcon \
-m 4096 \
-kernel ~/images/agl/Image \
-append "console=ttyAMA0 root=/dev/vda" \
-display gtk,show-cursor=on \
-smp 4 \
-device virtio-gpu-pci,virgl=true

Any tips on how to track down where things are going wrong?
--
Alex Bennée


Alex Bennée
 

Gerd Hoffmann <kraxel@...> writes:

Hi,

[ 2.930300] [drm] virgl 3d acceleration not supported by host
Nope, not active.

-display gtk,show-cursor=on \
Needs -display gtk,gl=on for opengl support.
Awesome - even on TCG the display is now nice and snappy.

For bonus points:

The AGL panel display is rotated 90 degrees which is putting a bit of a
crink in my neck. Is their anyway to rotate the view port. I tried
inverting xres and yres but that didn't do anything.

-device virtio-gpu-pci,virgl=true
virgl is silently turned off in case opengl support is not available.
Ideally virgl should be OnOffAuto not bool, but I fear changing that
now (and start throwing errors on virgl=on if opengl is not available)
will break stuff ...
At least a warn_report maybe?


take care,
Gerd

--
Alex Bennée


Marius Vlad
 

On Fri, Oct 23, 2020 at 12:52:17PM +0100, Alex Bennée wrote:

Gerd Hoffmann <kraxel@...> writes:

Hi,

[ 2.930300] [drm] virgl 3d acceleration not supported by host
Nope, not active.

-display gtk,show-cursor=on \
Needs -display gtk,gl=on for opengl support.
Awesome - even on TCG the display is now nice and snappy.

For bonus points:

The AGL panel display is rotated 90 degrees which is putting a bit of a
crink in my neck. Is their anyway to rotate the view port. I tried
inverting xres and yres but that didn't do anything.
Hi,

The output is rotated, edit /etc/xdg/weston/weston.ini and comment out
transform ini entry from the Virtual-1 output section. Reboot, or
restart weston@display service. Note that the apps are optimized for
landscape.

Enabling 3D with qemu might be something worth adding in the docs.

-device virtio-gpu-pci,virgl=true
virgl is silently turned off in case opengl support is not available.
Ideally virgl should be OnOffAuto not bool, but I fear changing that
now (and start throwing errors on virgl=on if opengl is not available)
will break stuff ...
At least a warn_report maybe?


take care,
Gerd

--
Alex Bennée





Alex Bennée
 

Thanks - that seems to do the trick.
I can't find where in the agl repo that is set. The recipe seems to be in:

meta-agl-cluster-demo/recipie-graphics/wayland/weston-ini-conf/virtual-landscape.cfg

but I can't see where the "transform=270" got added.

Visually it looks a lot nicer now although I could still do with
getting rid of the quite large AGL lower banner which is cropping out
the main display.

On Fri, 23 Oct 2020 at 13:32, Marius Vlad <marius.vlad@...> wrote:

On Fri, Oct 23, 2020 at 12:52:17PM +0100, Alex Bennée wrote:

Gerd Hoffmann <kraxel@...> writes:

Hi,

[ 2.930300] [drm] virgl 3d acceleration not supported by host
Nope, not active.

-display gtk,show-cursor=on \
Needs -display gtk,gl=on for opengl support.
Awesome - even on TCG the display is now nice and snappy.

For bonus points:

The AGL panel display is rotated 90 degrees which is putting a bit of a
crink in my neck. Is their anyway to rotate the view port. I tried
inverting xres and yres but that didn't do anything.
Hi,

The output is rotated, edit /etc/xdg/weston/weston.ini and comment out
transform ini entry from the Virtual-1 output section. Reboot, or
restart weston@display service. Note that the apps are optimized for
landscape.

Enabling 3D with qemu might be something worth adding in the docs.

-device virtio-gpu-pci,virgl=true
virgl is silently turned off in case opengl support is not available.
Ideally virgl should be OnOffAuto not bool, but I fear changing that
now (and start throwing errors on virgl=on if opengl is not available)
will break stuff ...
At least a warn_report maybe?


take care,
Gerd

--
Alex Bennée







--
Alex Bennée
KVM/QEMU Hacker for Linaro


Marius Vlad
 

On Fri, Oct 23, 2020 at 05:28:22PM +0100, Alex Bennée wrote:
Thanks - that seems to do the trick.
I can't find where in the agl repo that is set. The recipe seems to be in:

meta-agl-cluster-demo/recipie-graphics/wayland/weston-ini-conf/virtual-landscape.cfg

but I can't see where the "transform=270" got added.
I assume from
meta-agl/meta-agl-bsp/meta-aglprofilegraphical/recipes-graphics/wayland/weston-ini-conf/virtual.cfg

Visually it looks a lot nicer now although I could still do with
getting rid of the quite large AGL lower banner which is cropping out
the main display.
Yes, like I've said, but apparently I've substituted landscape w/
portrait, applications are designed to accommodate portrait orientation.
The panels, are added dynamically by homescreen.

On Fri, 23 Oct 2020 at 13:32, Marius Vlad <marius.vlad@...> wrote:

On Fri, Oct 23, 2020 at 12:52:17PM +0100, Alex Bennée wrote:

Gerd Hoffmann <kraxel@...> writes:

Hi,

[ 2.930300] [drm] virgl 3d acceleration not supported by host
Nope, not active.

-display gtk,show-cursor=on \
Needs -display gtk,gl=on for opengl support.
Awesome - even on TCG the display is now nice and snappy.

For bonus points:

The AGL panel display is rotated 90 degrees which is putting a bit of a
crink in my neck. Is their anyway to rotate the view port. I tried
inverting xres and yres but that didn't do anything.
Hi,

The output is rotated, edit /etc/xdg/weston/weston.ini and comment out
transform ini entry from the Virtual-1 output section. Reboot, or
restart weston@display service. Note that the apps are optimized for
landscape.

Enabling 3D with qemu might be something worth adding in the docs.

-device virtio-gpu-pci,virgl=true
virgl is silently turned off in case opengl support is not available.
Ideally virgl should be OnOffAuto not bool, but I fear changing that
now (and start throwing errors on virgl=on if opengl is not available)
will break stuff ...
At least a warn_report maybe?


take care,
Gerd

--
Alex Bennée








--
Alex Bennée
KVM/QEMU Hacker for Linaro