Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vulkan support [APK included] #596

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Vulkan support [APK included] #596

wants to merge 4 commits into from

Conversation

sfan5
Copy link
Member

@sfan5 sfan5 commented Sep 6, 2022

Opening this PR to keep track of the required changes to merge Vulkan support.

Checklist:

  • Figure out if we can avoid bumping the API minimum¹ (no)
  • Upstream fixes so that source patching is not needed anymore
  • Config UI to switch between opengl/vulkan
  • There should be a check whether vulkan works before users can enable it

¹ Raising the API to 24 would lose us 1.0% of our Google Play install base.

APK ->

@sfan5 sfan5 added the feature label Sep 6, 2022
@sfan5 sfan5 changed the title Vulkan support Vulkan support [APK included] Sep 7, 2022
@sfan5
Copy link
Member Author

sfan5 commented Sep 7, 2022

Added an APK for people wanting to test. It defaults to vo_gpu + Vulkan, vo_gpu_next is present too (both vk and gl) and you can switch by editing mpv.conf as usual.

@eko5624
Copy link

eko5624 commented Sep 8, 2022

Added an APK for people wanting to test. It defaults to vo_gpu + Vulkan, vo_gpu_next is present too (both vk and gl) and you can switch by editing mpv.conf as usual.

Device: HuaWei Mate 20 pro
OS: harmonyOS 2.0

MPV crashed when open a file. My mpv.config as below:

vo=gpu
gpu-context=auto
gpu-api=vulkan
hwdec=mediacodec-copy

@sfan5
Copy link
Member Author

sfan5 commented Sep 8, 2022

I can only diagnose issues with logcat output.

@eko5624
Copy link

eko5624 commented Sep 8, 2022

I can only diagnose issues with logcat output.

mpv.log

@sfan5
Copy link
Member Author

sfan5 commented Sep 8, 2022

Looks like you're missing support for timeline semaphores which means your phone/GPU is not supported.

@eko5624
Copy link

eko5624 commented Sep 8, 2022

Ok, thanks for your explaining.

@rohitghali
Copy link

Any updates on this? And, api is already bumped up right? Can strike off the first check box.

@sfan5
Copy link
Member Author

sfan5 commented Oct 24, 2022

I will revisit this sometime to work on the rest.
The purpose of the first box is to find out if we can get Vulkan working on newer devices and keep the API level the same.

@xdilemite
Copy link

I'm looking forward to getting a new modern build once the vulkan issue is fixed. Here's an old debug build from the source with working vulkan.mpv_vukan.apk

@rohitghali
Copy link

I'm looking forward to getting a new modern build once the vulkan issue is fixed. Here's an old debug build from the source with working vulkan.mpv_vukan.apk

How old is this?

@xdilemite
Copy link

I'm looking forward to getting a new modern build once the vulkan issue is fixed. Here's an old debug build from the source with working vulkan.mpv_vukan.apk

How old is this?

2019 but vulkan works.

@sfan5
Copy link
Member Author

sfan5 commented Jan 14, 2023

Vulkan fails to initialize

No surprise, this project might be able to work around the requirement in the future but I haven't investigated that yet.

OpenGL ES seems to be completely broken

I can't see any settings being applied in the log but you probably forgot to set gpu-context=android

mpv-android-2023-01-14_2.apk work again

Again no surprise as this is a totally regular build.

@sfan5
Copy link
Member Author

sfan5 commented Jan 16, 2023

gpu-api=opengl & gpu-context=android works for me here.

@sommio
Copy link

sommio commented Apr 11, 2023

Thankfully, it is more stable than master when I use vo=gpu-next & gpu-context=android & gpu-api=opengl in Kirin 810 + EMUI 12 (Android 10), although it doesn't work when I use vo=gpu-next & gpu-context=auto & gpu-api=vulkan

I can't even apply scale=lanzous and deband=yes for master, I highly recommend using vo=gpu-next & gpu-context=android & gpu-api=opengl in master by default

@hooke007 hooke007 mentioned this pull request Apr 16, 2023
@syphyr
Copy link
Contributor

syphyr commented May 22, 2023

When building libplacebo, the build fails when the android toolchain tries to include <shaderc/shaderc.h>. This fixes the issue: syphyr@b8c7260

All of the other pkg-config files have a prefix of "/usr/local". The shaderc pkg-config was the only one with a prefix of "/usr". There must have been some conflict with the Android toolchain and the search path.

@syphyr
Copy link
Contributor

syphyr commented May 22, 2023

Even though vulkan does not seem to be supported on my older Android tablet, I also wanted to mention that vo=gpu-next seems to work better than vo=gpu.

@sommio sommio mentioned this pull request May 23, 2023
@syphyr
Copy link
Contributor

syphyr commented May 23, 2023

There are actually two separate issues i encountered when building libplacebo.

The first issue is that $prefix_dir/lib does not get created for shaderc, so it fails copying libs.

The second issue is with the path inside the shaderc pkgconfig file mentioned previously.

Edit: my build machine does not have any local shaderc.h files, so maybe that is why I'm noticing this problem with the pkgconfig file.

@sfan5
Copy link
Member Author

sfan5 commented May 23, 2023

I tested this branch just yesterday and there were no issues with shaderc.
libplacebo did not want to build however as it requires Vulkan loader 1.3.0. With that patched up I found out that Vulkan doesn't work on my device as it requires 1.2.0 minimum.
Very few Android devices have that at all so I'm kind of doubting how much sense trying to support Vulkan makes.

@syphyr
Copy link
Contributor

syphyr commented May 23, 2023

I tested this branch just yesterday and there were no issues with shaderc. libplacebo did not want to build however as it requires Vulkan loader 1.3.0. With that patched up I found out that Vulkan doesn't work on my device as it requires 1.2.0 minimum. Very few Android devices have that at all so I'm kind of doubting how much sense trying to support Vulkan makes.

The issue i had with the include path for shaderc.h might be because of a difference with my build environment.

I also just started noticing that vo=gpu-next is causing random freezing in the video playback, but the audio keeps going.

@alakittty
Copy link

@sfan5 the shaderrc problem is there when building with the github actions build, but is fixed with syphrs patch, yesterday i forked mpv-android to do some dilly-dallying.

Btw how to build arm64v8a with gh actions?

@syphyr
Copy link
Contributor

syphyr commented May 24, 2023

Btw how to build arm64v8a with gh actions?

It looks like you need to add a line before this one to build arm64 variant.

./buildall.sh --arch arm64 mpv

./buildall.sh -n mpv || {

@syphyr
Copy link
Contributor

syphyr commented Jun 1, 2023

When building libplacebo separately, the build fails when shaderc attempts to copy libs to $prefix_dir. This fixes the issue:

syphyr@0825121

The Android NDK r25c contains shaderc v2022.1. The official Google shaderc repo has updated to v2023.5. This commit builds libplacebo with latest official google shaderc repo with updated 3rdparty libs. Also, using the google shaderc repo allows for including faster bugfixes and pinning versions.

syphyr@46f92aa

Edit: Updated links to include fixing copying shaderc header files. When copy -r was used, the shaderc header files were not getting copied into the correct subdirectory.

@alakittty
Copy link

Since this pr also seems to involve gpu-next, i bring this issue to the notice of the devs :

On my Samsung t505 tablet, with the given apk in the first comment here and also the apk built from syphrs' fork which uses latest versions of many deps, with the following config

vo=gpu-next
gpu-api=opengl
gpu-context=android

I notice very high number of frame drops with hwdec=auto(defaults to mediacodec), lesser drops with hwdec=no, and least with hwdec=auto-copy(mediacodec-copy), note that my device supports hardware vp9 decoding, but even with mediacodec-copy im getting 200+ framedrops for a 100 sec video. The video in question is here
also note that the stats don't show new frames/average frames/ mean frames

@alakittty
Copy link

Does A-V desync appear when framedrop=no?

no, there is no audio video desync with the config i mentioned above, but using framedrop=no introduces noticeable desync

Does this problem also happen with vo=gpu?

yes it does, but the order of framedrops is different mediacodec-copy>mediacodec>software

What is the hardware decoder throughput of your device?
ffmpeg -init_hw_device mediacodec=mc,create_window=1 -hwaccel mediacodec -i VIDEO -an -f null -
ffmpeg -init_hw_device mediacodec=mc,create_window=0 -hwaccel mediacodec -i VIDEO -an -f null -

do i use termux to execute that?

ps - i forgot to mention that this issue is non-existant with the stock play store release with no config

@alakittty
Copy link

alakittty commented Aug 6, 2023

Does this version manually set hwdec=mediacodec(-copy) also drop frames?

actually nvm it's just as bad with this version, also i don't believe using termux's version of ffmpeg to test out regression is plausible

@Jamie666123
Copy link

Since this pr also seems to involve gpu-next, i bring this issue to the notice of the devs :

On my Samsung t505 tablet, with the given apk in the first comment here and also the apk built from syphrs' fork which uses latest versions of many deps, with the following config

vo=gpu-next
gpu-api=opengl
gpu-context=android

I notice very high number of frame drops with hwdec=auto(defaults to mediacodec), lesser drops with hwdec=no, and least with hwdec=auto-copy(mediacodec-copy), note that my device supports hardware vp9 decoding, but even with mediacodec-copy im getting 200+ framedrops for a 100 sec video. The video in question is here also note that the stats don't show new frames/average frames/ mean frames

Where and how did you find this fork version of mpv android which includes gpu-next?

@obscenelyvague
Copy link

mpvlog.txt
mediacodec does not work with gpu-next and GPU API set to OpenGL. Shows solid purple color. Works with gpu
Is there a quality difference between mediacodec and mediacodec-copy?

@sfan5
Copy link
Member Author

sfan5 commented Aug 18, 2023

mediacodec does not work with gpu-next and GPU API set to OpenGL. Shows solid purple color.

Probably has to be fixed inside mpv, small issue.

Is there a quality difference between mediacodec and mediacodec-copy?

mediacodec-copy can be higher quality than mediacodec since it leaves the RGB conversion to mpv (whether this matters depends on the driver).
See also https://mpv.io/manual/master/#options-hwdec the "Quality reduction with [...]" section.

@obscenelyvague
Copy link

Okay, this is a weird one. I encountered a problem that one of the scalers may have helped catch. On the stable version, ewa_lanczos looked funky and I didn't want to bother tuning it so I just ended up using spline36. With gpu-next however, I thought the defaults for ewa_lanczos looked "fine", that is until I opened an external subtitle file for a video and immediately noticed the quality change. What I later discovered is upon first opening a video with mpv, the scaler probably functions as expected, but if you minimize the app into recents and go back OR do something from within the app that launches another activity, opening a subtitle file for example - the scaler behavior changes. See this screenshot comparison.
It's pretty easy to notice the scaler artifacts on the second pic.

vo=gpu-next profile=gpu-hq gpu-api=opengl gpu-context=android hwdec=no scale=ewa_lanczos
mpvlogscalerfinal.txt

@sfan5
Copy link
Member Author

sfan5 commented Sep 1, 2023

Looks like either some sort of GPU (driver) bug or a bug in libplacebo, that was probably already fixed in the meantime since the APK in OP is so outdated...

@obscenelyvague
Copy link

I'm keen on testing if a new APK is available with updated libplacebo.

@sfan5
Copy link
Member Author

sfan5 commented Sep 2, 2023

There's one now.

@obscenelyvague
Copy link

Hi, thanks for the new build. Unfortunately, the thing with scalers is still there. I can live with not minimizing the app and getting around having external subtitles by renaming them to be the same as video files. The more pressing issue seems to be the loss of video output (screen goes purple) after 10+ minutes of playback. Looking around, I found a similar issue that was filed here mpv-player/mpv#11721
But there are some key differences.
• I'm able to reproduce this behavior regardless of the codec being used, not just H.264
• I'm using SW decode and not mediacodec-copy
• Looking into logcat, I don't have those Adreno/KGSL errors
As for similarities
[vo/gpu-next/libplacebo:v] Detected fps ratio 0.0000 below threshold 0.0100, disabling interpolation
[cplayer:warn] Audio device underrun detected.
My log also has these two lines. That's apparently where the video freezes.

However, something else that may be of interest that is present in my log is this
[vo/gpu-next/libplacebo] gl_tex_upload: OpenGL error: GL_OUT_OF_MEMORY

Not sure if these two issues are actually the same as the original is referring to a video "freeze" whereas here, the screen goes purple.

Full log:
mpvlogbork2.txt

@sfan5
Copy link
Member Author

sfan5 commented Oct 9, 2023

Feel free to report that to haasn then, sounds like a real issue.

New build uploaded btw.

@obscenelyvague
Copy link

Previously it seemed like the audio underrun issue somehow led to the OpenGL OOM error but that might be a red herring because the audio underrun doesn't occur if speakers are used, only during USB-out. It probably has something to do with the device's audio HAL:

2023-10-13 04:20:06.133  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562812) frames_by_time(534) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.133  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562813) frames_by_time(524) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.134  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562814) frames_by_time(1021) > out->last_fifo_frames_remaining(1008)
2023-10-13 04:20:06.136  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562815) frames_by_time(525) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.142  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.142  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562816) frames_by_time(120) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.143  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562817) frames_by_time(620) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.143  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562818) frames_by_time(515) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.145  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.145  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562819) frames_by_time(90) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.145  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.146  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562820) frames_by_time(426) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.146  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562821) frames_by_time(544) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.148  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562822) frames_by_time(1060) > out->last_fifo_frames_remaining(1008)
2023-10-13 04:20:06.152  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.152  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562823) frames_by_time(103) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.153  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562824) frames_by_time(611) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.153  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562825) frames_by_time(554) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.155  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562826) frames_by_time(1190) > out->last_fifo_frames_remaining(1008)
2023-10-13 04:20:06.156  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.157  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562827) frames_by_time(452) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.157  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562828) frames_by_time(569) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.158  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562829) frames_by_time(1078) > out->last_fifo_frames_remaining(1008)
2023-10-13 04:20:06.162  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!
2023-10-13 04:20:06.162  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562830) frames_by_time(91) > out->last_fifo_frames_remaining(0)
2023-10-13 04:20:06.163  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562831) frames_by_time(579) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.165  1041 30552 audioserver W audio_hw_primary: out_write: underrun(562832) frames_by_time(537) > out->last_fifo_frames_remaining(504)
2023-10-13 04:20:06.165  1041 30552 audioserver W audio_hw_primary: out_get_presentation_position: avail:1008 > kernel_buffer_size:1008 clamping!

I've previously had an app that would cause these warnings to show up when AudioTrack API was selected but not for AAudio or OpenSL. In any case, it's probably not really something worth looking into if the warning is actually harmless.

I'm opening an issue on the mpv repo about the video playback issue. Not sure if the libplacebo repo would've been more appropriate but I'm hoping there'd be more eyes on the thing that way.

Btw, this is quite spammy on the last build:

[vo/gpu-next/libplacebo] libplacebo compiled without LittleCMS 2 support!

@syphyr
Copy link
Contributor

syphyr commented Oct 23, 2023

Since you just added libplacebo to the regular build, isn't shaderc a dependency of libplacebo?

@sfan5
Copy link
Member Author

sfan5 commented Oct 24, 2023

Only for Vulkan support.

@kasper93
Copy link

kasper93 commented Nov 26, 2023

Needs to make sure Vulkan is actually enabled.

diff --git a/buildscripts/scripts/mpv.sh b/buildscripts/scripts/mpv.sh
index feb85c9..c3e097f 100755
--- a/buildscripts/scripts/mpv.sh
+++ b/buildscripts/scripts/mpv.sh
@@ -18,7 +18,7 @@ unset CC CXX # meson wants these unset
 meson setup $build --cross-file "$prefix_dir"/crossfile.txt \
        --default-library shared \
        -Diconv=disabled -Dlua=enabled \
-       -Dlibmpv=true -Dcplayer=false \
+       -Dlibmpv=true -Dcplayer=false -Dvulkan=enabled \
        -Dmanpage-build=disabled

 ninja -C $build -j$cores

and also in mpv, to make Vulkan context available. There is no pc for vulkan so it fallback to system detection and this doesn't have version. Could generate custom .pc to workaround.

diff --git a/meson.build b/meson.build
index 0ff16b42d9..75b0cc89bb 100644
--- a/meson.build
+++ b/meson.build
@@ -1248,7 +1248,7 @@ vulkan_opt = get_option('vulkan').require(
     libplacebo.get_variable('pl_has_vulkan', default_value: '0') == '1',
     error_message: 'libplacebo compiled without vulkan support!',
 )
-vulkan = dependency('vulkan', version: '>= 1.1.70', required: vulkan_opt)
+vulkan = dependency('vulkan', required: vulkan_opt)
 features += {'vulkan': vulkan.found()}
 if features['vulkan']
     dependencies += vulkan

@syphyr
Copy link
Contributor

syphyr commented Nov 26, 2023

Enabling vulkan on api <= 24 will cause segfault at runtime.

@sfan5
Copy link
Member Author

sfan5 commented Nov 26, 2023

Could generate custom .pc to workaround.

I used to do this to make libplacebo happy but then it was no longer necessary.
mpv only needs it since mpv-player/mpv@662650b.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet