GNOME Bugzilla – Bug 781994
fixxref mixes and matches strings and bytes
Last modified: 2017-06-02 13:16:31 UTC
I get this trace when building gst-plugins-base: Error in gtkdoc helper script: 'gtkdoc-fixxref' failed with status 1 Traceback (most recent call last):
+ Trace 237412
fixxref.Run(options)
FixCrossReferences(options)
FixHTMLFile(options, full_entry)
repl_func, content, flags=re.DOTALL)
return _compile(pattern, flags).sub(repl, string, count)
return HighlightSource(options, m.group(1), m.group(2))
f.write(source)
return func(*args, **kwargs)
--- After fixing that one I get this: Error in gtkdoc helper script: 'gtkdoc-fixxref' failed with status 1 Traceback (most recent call last): File "/home/ernestas/jhbuild/install/bin/gtkdoc-fixxref", line 59, in <module> fixxref.Run(options) File "/home/ernestas/jhbuild/install/share/gtk-doc/python/gtkdoc/fixxref.py", line 113, in Run FixCrossReferences(options) File "/home/ernestas/jhbuild/install/share/gtk-doc/python/gtkdoc/fixxref.py", line 216, in FixCrossReferences FixHTMLFile(options, full_entry) File "/home/ernestas/jhbuild/install/share/gtk-doc/python/gtkdoc/fixxref.py", line 239, in FixHTMLFile repl_func, content, flags=re.DOTALL) File "/usr/lib/python3.6/re.py", line 191, in sub return _compile(pattern, flags).sub(repl, string, count) File "/home/ernestas/jhbuild/install/share/gtk-doc/python/gtkdoc/fixxref.py", line 236, in repl_func return HighlightSource(options, m.group(1), m.group(2)) File "/home/ernestas/jhbuild/install/share/gtk-doc/python/gtkdoc/fixxref.py", line 351, in HighlightSource highlighted_source = re.sub(r'^<\!-- .*? -->', '', highlighted_source, flags=re.MULTILINE | re.DOTALL) File "/usr/lib/python3.6/re.py", line 191, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: cannot use a string pattern on a bytes-like object
Created attachment 350774 [details] [review] fixxref: don’t mix and match bytes and strings NamedTemporaryFile() opens the file in binary mode by default, which doesn’t work, since strings are being written to it. Additionally, the code tries to perform a regex substitution on subprocess.check_output() return value, which is not a string. Signed-off-by: Ernestas Kulik <ernestask@gnome.org>
Patch tested only to the extent of making sure that building docs does not make the script throw a fit.
Review of attachment 350774 [details] [review]: Thanks!
Attachment 350774 [details] pushed as 7946cdf - fixxref: don’t mix and match bytes and strings
Attachment 350774 [details] seems to cause build failure for glib: DOC Fixing cross-references html/glib-Lexical-Scanner.html:1191: warning: no link for: "G-TOKEN-LAST:CAPS" -> (<code class="literal">G_TOKEN_LAST</code>). Traceback (most recent call last):
+ Trace 237470
Makefile:944: recipe for target 'html-build.stamp' failed
(In reply to Ting-Wei Lan from comment #5) > Attachment 350774 [details] seems to cause build failure for glib: > > DOC Fixing cross-references > html/glib-Lexical-Scanner.html:1191: warning: no link for: > "G-TOKEN-LAST:CAPS" -> (<code class="literal">G_TOKEN_LAST</code>). > Traceback (most recent call last): I’ll see what I can do about it.
Reverted the commit. It’s encodings all the way down. Forcing utf-8 for all files (codecs.open()) sort of works, but then it breaks with Python 3, because you can’t specify the encoding for temp files in Python 2 and we’re back at square one. A bit too much for one evening.
(In reply to Ting-Wei Lan from comment #5) > Attachment 350774 [details] seems to cause build failure for glib: > > DOC Fixing cross-references > html/glib-Lexical-Scanner.html:1191: warning: no link for: > "G-TOKEN-LAST:CAPS" -> (<code class="literal">G_TOKEN_LAST</code>). > Traceback (most recent call last): Are you sure that it’s my patch, though? It’s broken for me either way with Python 3.
gtkdoc-fixxref uses python 2.7.13 on my system because python is a symlink to python2. I saw the problem with commit 7946cdf, and 'git checkout' to the previous commit fixed it.
I'll take a look.
Created attachment 352730 [details] [review] don’t mix and match bytes and strings New attempt. This also works for python2. Ernestas, could you check if this still makes things work for you?
(In reply to Stefan Sauer (gstreamer, gtkdoc dev) from comment #11) > Created attachment 352730 [details] [review] [review] > don’t mix and match bytes and strings > > New attempt. This also works for python2. Ernestas, could you check if this > still makes things work for you? I tried, but bug 783178 happened. :|
Ernestas, I've pushed a fix for that. I need to figure a way to test with both python2 and python3 ...
I am now working with two checkout one configurred for py2 and one for py3. Will take care of this.
commit b9dfd5e2135fb84579c8c5971419b920bedd5bfc Author: Ernestas Kulik <ernestask@gnome.org> Date: Sun May 28 20:02:29 2017 +0200 fixxref: fix bytestring handling for python 3 NamedTemporaryFile() opens the file in binary mode by default, which doesn’t work, since strings are being written to it. Additionally, the code tries to perform a regex substitution on subprocess.check_output() return value, which is not a string.
All the recent fixes fixed the issue for me. Thanks!
Thanks for testing, really appreciated!
gtkdoc-fixxref failed when building gtk3: Makefile:968: recipe for target 'html-build.stamp' failed gmake: [html-build.stamp] Error 1 (ignored) gtkdoc-fixxref --module=gdk3 --module-dir=html --html-dir=/home/lantw44/gnome/devinstall/share/gtk-doc/html --extra-dir=/home/lantw44/gnome/devinstall/share/gtk-doc/html/gobject --extra-dir=/home/lantw44/gnome/devinstall/share/gtk-doc/html/glib --extra-dir=/usr/share/gtk-doc/html/cairo html/gdk3-Visuals.html:389: warning: no link for: "api-index-2.22" -> (2.22). html/gdk3-Visuals.html:869: warning: no link for: "api-index-2.2" -> (2.2). html/GdkScreen.html:466: warning: no link for: "api-index-2.8" -> (2.8). html/GdkScreen.html:499: warning: no link for: "api-index-2.10" -> (2.10). html/GdkScreen.html:896: warning: no link for: "api-index-2.20" -> (2.20). html/GdkScreen.html:1137: warning: no link for: "api-index-2.14" -> (2.14). html/GdkScreen.html:1348: warning: no link for: "PangoContext" -> (<span class="type">PangoContext</span>). html/GdkScreen.html:1413: warning: no link for: "PangoFontDescription" -> (<span class="type">PangoFontDescription</span>). html/gdk3-Cairo-Interaction.html:170: warning: no link for: "GdkPixbufs" -> (<span class="type">GdkPixbufs</span>). html/gdk3-Cairo-Interaction.html:532: warning: no link for: "GdkPixbuf" -> (<span class="type">GdkPixbuf</span>). html/gdk3-Cairo-Interaction.html:537: warning: no link for: "CAIRO-EXTEND-NONE:CAPS" -> (<code class="literal">CAIRO_EXTEND_NONE</code>). html/gdk3-Cairo-Interaction.html:632: warning: no link for: "api-index-2.24" -> (2.24). html/gdk3-Cairo-Interaction.html:786: warning: no link for: "GL-RENDERBUFFER:CAPS" -> (<span class="type">GL_RENDERBUFFER</span>). html/gdk3-Cairo-Interaction.html:787: warning: no link for: "GL-TEXTURE:CAPS" -> (<span class="type">GL_TEXTURE</span>). html/gdk3-Threads.html:226: warning: no link for: "G-THREADS-ENABLED:CAPS" -> (<span class="type">G_THREADS_ENABLED</span>). html/gdk3-Threads.html:337: warning: no link for: "api-index-2.4" -> (2.4). html/gdk3-Threads.html:376: warning: no link for: "api-index-2.12" -> (2.12). Traceback (most recent call last):
+ Trace 237537
Makefile:968: recipe for target 'html-build.stamp' failed gmake: *** [html-build.stamp] Error 1