GNOME Bugzilla – Bug 795792
cerbero backtrace when attempting to package GStreamer master build on ubuntu
Last modified: 2018-11-03 10:23:03 UTC
Cerbero crashes with a Python backtrace when it gets to the packaging stage on Ubuntu. Tested on Ubuntu 16.04 LTS and 18.04 LTS native and in docker containers. This appears to be an issue related to the porting of cerbero from Python 2 to Python 3 as it does not occur if I checkout version 1.12.4 of cerbero instead of master. The two warnings and the backtrace are: WARNING: No specific packager available for the distro version ubuntu_18_04_bionic, using generic packager for distro debian WARNING: No packager defined, using default packager "Default <default@change.me>" Traceback (most recent call last):
+ Trace 238598
main()
Main(sys.argv[1:])
self.run_command()
res = commands.run(command, self.config, self.args)
return _commands[command].run(config, args)
pkg = Packager(config, p, self.store)
return _packagers[d][v](config, package, store)
return DebianPackager(config, package, store)
self.license = f.read()
return codecs.ascii_decode(input, self.errors)[0]
Steps to reproduce in a docker container are: docker run -it ubuntu bash apt install -y sudo python3 python-dev git intltool populate your ~/.gitconfig for your identity mkdir ~/git cd ~/git git clone git://anongit.freedesktop.org/gstreamer/cerbero cd ~/git/cerbero ./cerbero-uninstalled bootstrap ./cerbero-uninstalled package gstreamer-1.0
Looking a bit further I see that class class DebianPackager in 'cerbero/packages/debian.py' reads from the file 'license.txt' which appears to be the file 'packages/gstreamer-1.0/license.txt'. This file does start with three strange characters: ~/git/cerbero# hexdump -C -n 14 packages/gstreamer-1.0/license.txt 00000000 ef bb bf 49 6e 73 74 61 6c 6c 65 72 2c 20 |...Installer, | 0000000e
Stripping non-ascii characters from the file "packages/gstreamer-1.0/license.txt" allows pacakging using the default packager to complete. My build process is now: docker run -it ubuntu bash apt install -y sudo python3 python-dev git intltool populate your ~/.gitconfig for your identity mkdir ~/git cd ~/git git clone git://anongit.freedesktop.org/gstreamer/cerbero cd ~/git/cerbero ./cerbero-uninstalled bootstrap perl -pi -e 's/[^[:ascii:]]//g' packages/gstreamer-1.0/license.txt ./cerbero-uninstalled package gstreamer-1.0
As noted in the gstreamer-devel mailing list, the above steps are not totally accurate as I'm 1) restricting myself to the ubuntu:xenial docker image and I'm using the 1.14 branch of cerbero. So the exact steps to reproduce what I'm doing (and get packages built) are: 1. docker run -it ubuntu:xenial bash 2. apt install -y sudo python3 python-dev git intltool 3. populate your ~/.gitconfig for your identity 4. mkdir ~/git 5. cd ~/git 6. git clone git://anongit.freedesktop.org/gstreamer/cerbero -b 1.14 7. cd ~/git/cerbero 8. git checkout 1.14 9. yes | ./cerbero-uninstalled bootstrap 10. perl -pi -e 's/[^[:ascii:]]//g' packages/gstreamer-1.0/license.txt 11. ./cerbero-uninstalled package gstreamer-1.0
I don't think we can modify the license files in place as a permanent solution. There is nothing about Debian policy that prohibits UTF-8 license files (see https://www.debian.org/doc/debian-policy/), so I don't think it's a Debian problem. It must be a python (cerbero) problem requiring a modification to the cerbero code.
Yes, the Python code need to support this.
I know that this problem did not exist in Cerbero 1.12.4. Although I haven't looked at the code yet, this is probably a result of the Python 2 --> 3 conversion for which there are many implications around string encoding. Fixing this is probably a simple matter of treating the contents of the license file as a byte array (thus avoiding any string conversions), but it may be more complicated then that.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/cerbero/issues/54.