GNOME Bugzilla – Bug 783926
gtkdoc-fixxref crashes when stdout is a not a terminal
Last modified: 2017-06-23 14:56:47 UTC
This problem can be easily reproduced by running 'make' with stdout redirected to another program. For example, if we send the content of stdout to tee when building GLib. $ gmake V=1 | tee /dev/null gmake[2]: [Makefile:945: html-build.stamp] Error 1 (ignored) gtkdoc-fixxref --module=glib --module-dir=html --html-dir=/home/lantw44/gnome/devinstall/share/gtk-doc/html --extra-dir=/home/lantw44/gnome/source/glib/docs/reference/glib/../gobject/html --extra-dir=/home/lantw44/gnome/source/glib/docs/reference/glib/../gio/html html/glib-Standard-Macros.html:571: warning: no link for: "G-DISABLE-CONST-RETURNS:CAPS" -> (<code class="literal">G_DISABLE_CONST_RETURNS</code>). html/glib-The-Main-Event-Loop.html:3162: warning: no link for: "GChildWatchSource" -> (<span class="type">GChildWatchSource</span>). html/glib-The-Main-Event-Loop.html:3393: warning: no link for: "fd" -> (<code class="literal">fd</code>). html/glib-The-Main-Event-Loop.html:3394: warning: no link for: "events" -> (<code class="literal">events</code>). html/glib-The-Main-Event-Loop.html:3395: warning: no link for: "revents" -> (<code class="literal">revents</code>). html/glib-Threads.html:1227: warning: no link for: "G-LOCK-:CAPS" -> (<span class="type">G_LOCK_</span>). html/glib-Threads.html:3204: warning: no link for: "pthread-setspecific" -> (<code class="function">pthread_setspecific()</code>). html/glib-Threads.html:3204: warning: no link for: "pthread-getspecific" -> (<code class="function">pthread_getspecific()</code>). html/glib-Dynamic-Loading-of-Modules.html:523: warning: no link for: "g-module-check-init" -> (<code class="function">g_module_check_init()</code>). html/glib-Dynamic-Loading-of-Modules.html:553: warning: no link for: "g-module-unload" -> (<code class="function">g_module_unload()</code>). html/glib-Memory-Slices.html:125: warning: no link for: "GMemChunks" -> (<span class="type">GMemChunks</span>). html/glib-Memory-Slices.html:135: warning: no link for: "posix-memalign" -> (<code class="function">posix_memalign()</code>). html/glib-Memory-Slices.html:361: warning: no link for: "G_DEBUG" -> (<code class="literal">G_DEBUG=gc-friendly</code>). html/glib-IO-Channels.html:2495: warning: no link for: "O-APPEND:CAPS" -> (<code class="literal">O_APPEND</code>). html/glib-IO-Channels.html:2504: warning: no link for: "O-NONBLOCK:CAPS" -> (<code class="literal">O_NONBLOCK</code>). html/glib-IO-Channels.html:2504: warning: no link for: "O-NDELAY:CAPS" -> (<code class="literal">O_NDELAY</code>). html/glib-Message-Logging.html:313: warning: no link for: "G-LOG-USE-STRUCTURED:CAPS" -> (<code class="literal">G_LOG_USE_STRUCTURED</code>). html/glib-Message-Logging.html:318: warning: no link for: "testing-for-messages" -> (Testing for Messages). html/glib-Message-Logging.html:439: warning: no link for: "using-structured-logging" -> (Using Structured Logging). html/glib-Message-Logging.html:887: warning: no link for: "g-log-sets-handler" -> (<code class="function">g_log_sets_handler()</code>). html/glib-String-Utility-Functions.html:1207: warning: no link for: ".codeset" -> (_territory). html/glib-String-Utility-Functions.html:1657: warning: no link for: "FILE:CAPS" -> (<span class="type">FILE</span>). html/glib-String-Utility-Functions.html:2074: warning: no link for: "EOF:CAPS" -> (<code class="literal">EOF</code>). html/glib-String-Utility-Functions.html:3150: warning: no link for: "HUGE-VAL:CAPS" -> (<code class="literal">HUGE_VAL</code>). html/glib-String-Utility-Functions.html:3151: warning: no link for: "ERANGE:CAPS" -> (<code class="literal">ERANGE</code>). html/glib-String-Utility-Functions.html:3152: warning: no link for: "errno" -> (<code class="literal">errno</code>). html/glib-Character-Set-Conversion.html:274: warning: no link for: "file-name-encodings-diagram" -> (diagram). html/glib-Character-Set-Conversion.html:612: warning: no link for: "iconv-open" -> (<code class="function">iconv_open()</code>). html/glib-Character-Set-Conversion.html:708: warning: no link for: "iconv-close" -> (<code class="function">iconv_close()</code>). html/glib-Unicode-Manipulation.html:1117: warning: no link for: "11" -> (<span class="type">11</span>). html/glib-Unicode-Manipulation.html:1353: warning: no link for: "15" -> (<span class="type">15</span>). html/glib-Unicode-Manipulation.html:4005: warning: no link for: "24" -> (<span class="type">24</span>). html/glib-I18N.html:141: warning: no link for: "GETTEXT-PACKAGE:CAPS" -> (<code class="literal">GETTEXT_PACKAGE</code>). html/glib-Date-and-Time-Functions.html:2429: warning: no link for: "GDate-struct" -> (<span class="type">GDTraceback (most recent call last): ate</span>). html/glib-Random-Numbers.html:314: warning: no link for: "rand-s" -> (<code class="function">rand_s()</code>).
+ Trace 237581
fixxref.Run(options)
FixCrossReferences(options)
FixHTMLFile(options, full_entry)
lines[i] = re.sub(r'<GTKDOCLINK\s+HREF="([^"]*)"\s*>(.*?)</GTKDOCLINK\s*>', repl_func_with_ix(i), lines[i])
return _compile(pattern, flags).sub(repl, string, count)
return MakeXRef(options, file, i + 1, m.group(1), m.group(2))
common.LogWarning(file, line, 'no link for: "%s" -> (%s).' % (id, text))
print ("%s:%d: warning: %s" % (filename, line, message))
gmake[2]: *** [Makefile:946: html-build.stamp] Error 1 gmake[2]: Leaving directory '/home/lantw44/gnome/build/glib/docs/reference/glib' gmake[1]: *** [Makefile:487: all-recursive] Error 1 gmake[1]: Leaving directory '/home/lantw44/gnome/build/glib/docs/reference' gmake: *** [Makefile:487: all-recursive] Error 1 This problem can also happen when using 'meson' or 'jhbuild tinderbox' because they redirect stdout to a pipe. jhbuild tinderbox log: https://jhbuild.csie.org:9043/20170618/glib.html
This problem can only be reproduced with python2.
some possible solution in https://stackoverflow.com/questions/492483/setting-the-correct-encoding-when-piping-stdout-in-python need to test ...
As a workaround you should be able to run as PYTHONIOENCODING="utf-8" gmake V=1 | tee /dev/null
Created attachment 354150 [details] [review] handle encoding when redirecting stdout under python2 This works for me. It would be awesome if you could try and confirm. Thanks!
(In reply to Stefan Sauer (gstreamer, gtkdoc dev) from comment #4) > Created attachment 354150 [details] [review] [review] > handle encoding when redirecting stdout under python2 > > This works for me. It would be awesome if you could try and confirm. Thanks! Yes, it fixes 'jhbuild tinderbox glib' on my machine. I am running another rebuild to know whether it can cause other problems.
(In reply to Ting-Wei Lan from comment #5) > (In reply to Stefan Sauer (gstreamer, gtkdoc dev) from comment #4) > > Created attachment 354150 [details] [review] [review] [review] > > handle encoding when redirecting stdout under python2 > > > > This works for me. It would be awesome if you could try and confirm. Thanks! > > Yes, it fixes 'jhbuild tinderbox glib' on my machine. I am running another > rebuild to know whether it can cause other problems. Nice, I found no related build issues after applying the patch.
Thanks for testing. Pushed.