GNOME Bugzilla – Bug 737429
Android runtime contains libs with rpath
Last modified: 2018-11-03 10:17:53 UTC
When using the gstreamer Android runtime there are a few libraries that get blacklisted Total count: 157 plugins (34 blacklist entries not shown), 976 features A gst-inspect-1.0 x264enc shows: (gst-inspect-1.0:10797): GStreamer-WARNING **: Failed to load plugin '/storage/sdcard1/gstreamer/lib/gstreamer-1.0/libgstx264.so': dlopen failed: could not load library "libgstpbutils-1.0.so.0" needed by "libgstx264.so"; caused by could not load library "libgstaudio-1.0.so.0" needed by "libgstpbutils-1.0.so.0"; caused by could not load library "libgsttag-1.0.so.0" needed by "libgstaudio-1.0.so.0"; caused by could not load library "/home/slomo/Projects/gstreamer/cerbero/dist/android-nodebug/lib/libz.so" needed by "libgsttag-1.0.so.0"; caused by library "/home/slomo/Projects/gstreamer/cerbero/dist/android-nodebug/lib/libz.so" not found element plugin couldn't be loaded No such element or plugin 'x264enc' Using ndk-depends on libgsttag show that libz still has the rpath, when it shouldn't. I'm not sure if there are other libs with the same problem. C:\Android\android-ndk-r10>ndk-depends.exe C:\Users\ck \Downloads\gstreamer-1.0-android-arm-1.4.1-release-runtime\lib\libgsttag-1.0.so.0 WARNING: Could not find library: /home/slomo/Projects/gstreamer/cerbero/dist/android-nodebug/lib/libz.so libgsttag-1.0.so.0 libgstbase-1.0.so.0 libgstreamer-1.0.so.0 libm.so libgobject-2.0.so.0 libgmodule-2.0.so.0 libglib-2.0.so.0 libintl.so.8 libiconv.so.2 libffi.so.6 libdl.so libc.so /home/slomo/Projects/gstreamer/cerbero/dist/android-nodebug/lib/libz.so
Note, we don't usually test running non-static GStreamer on Android. The reason is that there is a limited about of shared object that can be opened (something around 25 per apps). Never the less, this cross-compilation issue would be nice to solve in Cerbero as this may affect other platforms.
*** Bug 738024 has been marked as a duplicate of this bug. ***
This limit of 64 is not true anymore with latest Android (I think it's something like 2**10 or 2**16 there). However we only really support static linking on Android and for everything else you're on your own and will likely run into problems. Nonetheless we should fix this rpath problem.
(In reply to Sebastian Dröge (slomo) from comment #3) > This limit of 64 is not true anymore with latest Android (I think it's > something like 2**10 or 2**16 there). However we only really support static > linking on Android and for everything else you're on your own and will > likely run into problems. > > Nonetheless we should fix this rpath problem. I have the same problem, have you fix the rpath problem. Thanks a lot.
*** Bug 748993 has been marked as a duplicate of this bug. ***
I ran into this as well. Upgrading from the gstreamer-sdk to the gstreamer version of cerbero fixed all the other rpath problems I was having except libz. The Android linker will emit a message such as "'libgiognutls.so' library has invalid DT_NEEDED entry '/home/andrewes/cerbero/dist/android_armv7/lib/libz.so'", but it appears to find and use the library anyway. Code which emits the linker message and corrects the path can be seen at the bottom of: https://android.googlesource.com/platform/bionic/+/c3351ea%5E!/ I got rid of the warning message by hand-editing the .so files that have the rpath entry by overwriting the first bytes of the path with "libz.so\0". Note that the NDK also provides a version of libz.so, which may be what is causing the build to try to specify a "private" version within the cerbero build. We need control over that path, since it is obviously wrong for Android.
I've made a little progress on this. The libz library is built within cerbero. It is also available from the NDK. When the linker find both of these, it determines that the developer is trying to use a "replacement" library for the "system" one in the NDK. It adds the hardcoded path so the executable will be certain to use the library the developer specified, even when a system one could be found. On Android, the "system" vs "replacement" library problem is solved by searching the APK path first, followed by the /system/lib path. All libraries must not have a path. The hardcoded path solution will fail on any cross development platform, to be honest. Pointing the dist/<platform>/lib/libz.la file's libdir entry to the NDK version of libz.so solves the problem in many cases. This causes the linker to see a match between the "system" library and the one being linked, so it does not add the path.
Do we even need to build libz on android if it's already provided by the platform?
There is a gap in the recipe language support: You can add to the deps list with platform_deps, but you can't subtract, or say "All platforms except this one". So someone would have to go through all the recipes and remove zlib from all the deps lists, and then add it back in to platform_deps for all the platforms except Android.
We have per-arch deps in packages/base-system-1.0.package -- would that not suffice?
Er, I mean per-platform of course.
Recipes also have zlib in their dependencies, and then link to our version of it. Apparently zlib is public API in the NDK, so we could indeed use the version from Android. However it's worrying that our version does not work properly, and I'd prefer if we also would fix that at the same time.
Isn't the reason it doesn't work for the very reason that there's a public version in the NDK?
I think what we're talking about is how we can stop including zlib in the Android build.
Why would it not work if the NDK includes it but we only statically link to it inside our libgstreamer_android.so?
The version of zlib in the Android I am targeting is 1.2.8, which is the same as what version 1.9.90 of cerbero is also building for me. There is a possibility that the ./configure stage configured them differently, but packages depending on zlib also have a config stage, so they should customize themselves to which ever zlib they are exposed to.
I am linking dynamically so I can keep my image small, and take advantage of system libraries like zlib. If I link statically, I bring in an extra copy of zlib.
Arun> "per-arch deps in packages/base-system-1.0.package" I have to study what is in "packages/*" and see what is going on there. I do my build by picking out individual recipes and building them directly. I don't know how to use packages yet, other than to list them. If I can create and "android" package that globally overrides the dependencies, that might suffice.
(In reply to Andrew Esh from comment #17) > I am linking dynamically so I can keep my image small We actually do static linking to keep the size small. The linker will drop everything that is not used.
For the shared library, couldn't we just remove the rpath from all of them as a post-build step for Android? There's chrpath for that for example.
Just to be clear, the RPATH setting is not the problem. In the dynamic table (seen with "objdump -p ..."), the soname in the NEEDED entry has been path-extended. The RPATH field is blank. So far, the only way I have found to remove the path is to hand-edit the binary with a hex editor.
I ran into this problem in a cerbero build which was getting an extended SONAME reference in libz.so during the zlib build. To solve it, I changed the m4 macro within the project directory prior to building anything that depended on libz. The change formats the soname so it does not collide with the system libz.so. NOTE: The each of the next globs of text are one line. sed -i "s/soname_spec='\$libname\$release\$shared_ext'/soname_spec='libqda\`echo \$name\$release\$shared_ext\`'/g" build/build-tools/share/aclocal/libtool.m4 $CERBERO_EXE -c config/$CONFIG buildone libjpeg-turbo Once libjpeg-turbo is built, and picks up the libqda version of the soname, I change libz back to what it was: sed -i "s/soname_spec='libqda\`echo \$name\$release\$shared_ext\`'/soname_spec='\$libname\$release\$shared_ext'/g" build/build-tools/share/aclocal/libtool.m4 $CERBERO_EXE -c config/$CONFIG build zlib Then, the libtool file is updated and copied into the sysroot to point all the other dependents to the altered library: NDK_SYSROOT=$ANDROID_NDK_ROOT-$ANDROID_NDK_ARCH-$ANDROID_NDK_API sed -i 's@^libdir.*@libdir='"$NDK_SYSROOT"'/sysroot/usr/lib@g' build/dist/$DISTDIR/lib/libz.la cp build/dist/$DISTDIR/lib/libz.la $NDK_SYSROOT/sysroot/usr/lib Because the new library name does not collide, the code which adds the full path to the soname is not triggered.
I just ran into the same problem while cross-compiling for an arm platform and it looks like the problem originates in that libz.so is being compiled without a SONAME in its headers. This is a regression introduced by 9afc668 - zlib: Use host if provided (see https://bugzilla.gnome.org/show_bug.cgi?id=768493). Apparently, the argument to --uname is used in zlib's configure to figure out whether the -soname argument should be passed to the linker or not: .... case "$uname" in Linux* | linux* | GNU | GNU/* | solaris*) LDSHARED=${LDSHARED-"$cc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map"} ;; .... and with uname being "arm-linux-gnueabihf" in my case, this code is never executed, so zlib is being compiled without a SONAME and this is why the linker generates dependencies with an absolute path. RPATH is not actually an issue.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/cerbero/issues/5.