After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 727587 - reproducible gnome-terminal-server crash on OpenBSD
reproducible gnome-terminal-server crash on OpenBSD
Status: RESOLVED FIXED
Product: vte
Classification: Core
Component: general
0.36.x
Other OpenBSD
: Normal normal
: ---
Assigned To: GNOME Terminal Maintainers
GNOME Terminal Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-04-04 08:36 UTC by Antoine Jacoutot
Modified: 2014-08-04 16:06 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
use sizeof (741 bytes, patch)
2014-04-04 16:18 UTC, Antoine Jacoutot
rejected Details | Review
print utf8 chars randomly (235 bytes, application/x-ruby)
2014-07-28 21:18 UTC, Robert Nagy
  Details
use sizeof v2 (817 bytes, text/plain)
2014-07-29 14:35 UTC, Antoine Jacoutot
  Details
clamp the end computation to the string length (812 bytes, patch)
2014-07-30 09:00 UTC, Antoine Jacoutot
none Details | Review
Fix (for vte 0.36) (6.07 KB, patch)
2014-08-01 09:51 UTC, Egmont Koblinger
none Details | Review
Fix v2 (for vte 0.36) (7.07 KB, patch)
2014-08-01 22:13 UTC, Egmont Koblinger
none Details | Review
Fix v2 (for vte 0.37) (7.28 KB, patch)
2014-08-01 22:29 UTC, Egmont Koblinger
none Details | Review
Fix v3 (for vte 0.36) (7.14 KB, patch)
2014-08-02 16:21 UTC, Egmont Koblinger
committed Details | Review
Fix v3 (for vte 0.37) (7.35 KB, patch)
2014-08-02 16:21 UTC, Egmont Koblinger
committed Details | Review

Description Antoine Jacoutot 2014-04-04 08:36:06 UTC
Hi.

I have a reproducible crash of gnome-terminal-server under OpenBSD.
I can easily trigger it by moving from one tmux window to another that contains utf8 chars.

0x00001d14407c9fbb in vte_terminal_accessible_get_text (text=0x1d1437ec06b0, start_offset=0, end_offset=3465) at vteaccess.c:1075
	priv = (VteTerminalAccessiblePrivate *) 0x1d14428ddcc0
	start = 0
	end = 4430
	ret = (gchar *) 0x1d145529c000 "������"
	__FUNCTION__ = "vte_terminal_accessible_get_text"


  • #0 memcpy
    at /usr/src/lib/libc/string/bcopy.c line 115
  • #1 vte_terminal_accessible_get_text
    at vteaccess.c line 1075
  • #2 text_changed_event_listener
    from /usr/local/lib/libatk-bridge-2.0.so.0.0
  • #3 signal_emit_unlocked_R
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #4 g_signal_emit_valist
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #5 g_signal_emit_by_name
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #6 emit_text_changed_delete
    at vteaccess.c line 234
  • #7 vte_terminal_accessible_text_modified
    at vteaccess.c line 513
  • #8 _g_closure_invoke_va
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #9 g_signal_emit_valist
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #10 g_signal_emit_by_name
    from /usr/local/lib/libgobject-2.0.so.4000.0
  • #11 _vte_terminal_emit_text_inserted
    at vte.c line 971
  • #12 vte_terminal_emit_pending_text_signals
    at vte.c line 4104
  • #13 vte_terminal_emit_pending_signals
    at vte.c line 14730
  • #14 vte_terminal_process_incoming
    at vte.c line 4488
  • #15 time_process_incoming
    at vte.c line 14741
  • #16 process_timeout
    at vte.c line 14788
  • #17 g_timeout_dispatch
    from /usr/local/lib/libglib-2.0.so.4000.0
  • #18 g_main_context_dispatch
    from /usr/local/lib/libglib-2.0.so.4000.0
  • #19 g_main_context_iterate
    from /usr/local/lib/libglib-2.0.so.4000.0
  • #20 g_main_context_iteration
    from /usr/local/lib/libglib-2.0.so.4000.0
  • #21 g_application_run
    from /usr/local/lib/libgio-2.0.so.4000.0
  • #22 main
    at server.c line 134

Comment 1 Christian Persch 2014-04-04 08:49:45 UTC
Please get a valgrind log of this.
Comment 2 Antoine Jacoutot 2014-04-04 08:55:12 UTC
(In reply to comment #1)
> Please get a valgrind log of this.

Hi Christian.

I wish but unfortunately valgrind is not available on OpenBSD :-/
Comment 3 Antoine Jacoutot 2014-04-04 13:23:16 UTC
Oh, just realized something... it looks like this is happening _only_ when gnome-terminal is run in full screen mode (f11) - sorry but since this is how I always run it, I missed to mention or even think about it.
Comment 4 Antoine Jacoutot 2014-04-04 13:38:52 UTC
(In reply to comment #3)
> Oh, just realized something... it looks like this is happening _only_ when
> gnome-terminal is run in full screen mode

Actually, not _only_ but having g-t running full screen makes the bug 100% reproducible. Otherwise in non full screen mode, it happens maybe once out of 15 times or so.
Comment 5 Antoine Jacoutot 2014-04-04 16:18:32 UTC
Created attachment 273594 [details] [review]
use sizeof

So far, this seems to do the trick for me; at least I haven't been able to trigger the crash yet.
Comment 6 Antoine Jacoutot 2014-04-04 16:36:39 UTC
And of course, just 10 minutes after writing this message, it crashed...
I am really lost :-/
Comment 7 Christian Persch 2014-04-05 08:18:54 UTC
Can you try recording the reproducer with script(1) and see if the typescript still reproduces (try with recording the timings too if it doesn't repro without it).
Comment 8 Antoine Jacoutot 2014-04-05 12:33:49 UTC
(In reply to comment #7)
> Can you try recording the reproducer with script(1) and see if the typescript
> still reproduces (try with recording the timings too if it doesn't repro
> without it).

Hi Christian. I am not sure what you mean by "see if the typescript still reproduces" -- but running under script(1) the crash happens in the same way.
Comment 9 Christian Persch 2014-04-05 13:26:36 UTC
Try if replaying the captured typescript reproduces, using e.g. cat(1) or scriptreplay(1) (for the timings).
Comment 10 Antoine Jacoutot 2014-04-05 13:47:16 UTC
(In reply to comment #9)
> Try if replaying the captured typescript reproduces, using e.g. cat(1) or
> scriptreplay(1) (for the timings).

Replaying the typescript does not reproduce the issue :-/
Comment 11 Christian Persch 2014-04-27 08:01:56 UTC
That's rather unexpected. Running under script simly makes a copy of the output, and when replayed with timing should make the terminal behave exactly as when running direct...
Comment 12 Antoine Jacoutot 2014-04-27 08:04:40 UTC
(In reply to comment #11)
> That's rather unexpected. Running under script simly makes a copy of the
> output, and when replayed with timing should make the terminal behave exactly
> as when running direct...

Yes I know, I am totally puzzled by this issue :-(
Comment 13 Egmont Koblinger 2014-05-28 17:38:15 UTC
Antoine:  Could you please give exact, step-by-step description of what you are doing?  I'd like to reproduce the bug, but "moving from one tmux window to another that contains utf8 chars" is way too vague for me.  Please be as specific as you can, mention every single keypress (I'm not that familiar with tmux and would prefer not go dig into its manual right now), mention the exact contents of the screen, etc.  Thanks!

I turned on the screen reader (according to your stack trace, some a11y stuff is required), started a maximized vte window, tmux, typed a few UTF-8 characters, pressed ^B C to create a new window, typed another few chars there, pressed ^B L to switch between them a couple of times – no crash yet.
Comment 14 Antoine Jacoutot 2014-05-30 09:20:27 UTC
(In reply to comment #13)
> Antoine:  Could you please give exact, step-by-step description of what you are
> doing?  I'd like to reproduce the bug, but "moving from one tmux window to
> another that contains utf8 chars" is way too vague for me.  Please be as

Yeah I understand it being vague but the bug is not 100% reproducible. It just happens (often).
Ok there is one situation that will trigger the segfault much more often than others. Being in tmux and moving from an irssi window into a calcurse one (http://calcurse.org/).
In tmux, moving from one windows to the other is done with ^b then X where X is the window number.

> specific as you can, mention every single keypress (I'm not that familiar with
> tmux and would prefer not go dig into its manual right now), mention the exact
> contents of the screen, etc.  Thanks!

The content is just irssi on any IRC channel and a defautl calcurse window...
But really it can also happen out of the blue without running tmux at *all*.
Something gnome-terminal-server crashes while displaying the output of a compilation.
So far I can only reproduce this on OpenBSD though (but I don't use Linux that often).
Comment 15 Egmont Koblinger 2014-05-30 09:47:40 UTC
I still can't reproduce.  (I only have Linux, though.)

How often does it occur to you?  How long should I try?

Could you maybe create a screencast (e.g. with kazam) and upload to youtube? There might be something you're doing differently which just wouldn't occur to me. (Make sure it doesn't contain any personal or sensitive info. You can send it to my email if you prefer not to share with others.)

Does this happen in standalone vte app? (Download vte; ./configure; make; ./src/vte[TAB])  If it does, does it also happen with vte git master?
Comment 16 Egmont Koblinger 2014-05-30 09:51:20 UTC
Review of attachment 273594 [details] [review]:

sizeof() is a compile-time thingy, its value is the size of a pointer in this case, either 4 or 8 depending on architecture. So this patch is definitely incorrect. :(
Comment 17 Antoine Jacoutot 2014-05-30 10:35:36 UTC
(In reply to comment #15)
> I still can't reproduce.  (I only have Linux, though.)
> 
> How often does it occur to you?  How long should I try?

It happens around 10 times per day.
And since it is the server part that crashes, I loose all my terms.
I am now using xterm(1) for important terminal apps, but meh... :-/

> Could you maybe create a screencast (e.g. with kazam) and upload to youtube?
> There might be something you're doing differently which just wouldn't occur to
> me. (Make sure it doesn't contain any personal or sensitive info. You can send
> it to my email if you prefer not to share with others.)

Well as mentioned several times, it's not reproducible all the time...
Sure I could try to record the crash but there is really nothing special.
irssi showing #gnome-hackers and default calcurse window. Really there is nothing special about it.

> Does this happen in standalone vte app? (Download vte; ./configure; make;
> ./src/vte[TAB])  If it does, does it also happen with vte git master?

No, I was never able to reproduce in the standalone vte term.
Comment 18 Christian Persch 2014-05-30 10:59:31 UTC
Can you please attach the typescript (with timing, and --flush so that everything is captured even when the crash occurs) of a reproducing run? Even if the replay doesn't crash for you, running it on linux with valgrind might still get some info possibly.
Comment 19 Egmont Koblinger 2014-05-30 11:38:36 UTC
We've just fixed a crash which occurs if you drag-n-drop a tab, and afterwards change that tab's title.  Is it maybe that you drag-n-drop a tab, and tmux updates the tab title when you change the tmux window?

Could you please attach your .tmux.conf file (if it exists)?
Comment 20 Antoine Jacoutot 2014-05-30 11:41:38 UTC
(In reply to comment #19)
> We've just fixed a crash which occurs if you drag-n-drop a tab, and afterwards
> change that tab's title.  Is it maybe that you drag-n-drop a tab, and tmux
> updates the tab title when you change the tmux window?

I dont' use tabs. Well I don't use vte/gnome-terminal tabs, I use tmux windows/panes.

> Could you please attach your .tmux.conf file (if it exists)?

No .tmux.conf file here. As mentioned, *nothing* special, nothing configured.
Comment 21 Egmont Koblinger 2014-05-30 12:30:05 UTC
> It happens around 10 times per day.

Well, I don't know how much use your computer per day :)

Let's imagine I sat next to you and you wanted to demonstrate the crash to me as quickly as you can.  You start up a gnome-terminal, tmux, two windows: irssi and calcurse.  You switch tmux window back and forth 5-10 times and it crashes?  Or a thousand times?  Would you need to actively use these apps?  Would I have to watch for a few seconds, minutes, hours until the crash?

> Sure I could try to record the crash but there is really nothing special.

There might be something special for us.  Maybe not.  We can't tell in advance.

> No, I was never able to reproduce in the standalone vte term.

What are the exact gnome-terminal and vte version numbers?

Could you please ensure that gnome-terminal and vte app are using the exact same libvte?  (Not just the same version, but actually the same file, to guarantee the same compiler flags, etc.)  Is it still the case that standalone vte doesn't crash, but gnome-terminal does?  [What I'm hoping for is that in this case either both crash or none of them, making it clear that the bug is in vte.]

I recommend you to vteapp for a while for non-critical tasks, first to see if it really doesn't crash, and second: even if it crashes, your other vteapp windows will stay alive which is still better than a gnome-terminal crash.

What C compiler do you have?

Does it always crash at the same location (vteaccess.c:1075)?
Comment 22 Antoine Jacoutot 2014-06-02 21:02:56 UTC
Hi Egmont.

Just a quick note to let you know that I did not forget about the issue. I have just been totally over busy lately but I will get back to you with the requested info as soon as I can...
Comment 23 Antoine Jacoutot 2014-06-13 08:21:51 UTC
Ok I have been doing more testings after moving the machine I was ssh'ing into from sparc64 to amd64 and it looks like the problem is gone. So it is possible the remote ssh connection window in tmux (sparc64) was sending garbled chars while with amd64 it doesn't.
I will keep testing this for a few days and if it still works without issue, I will close this bug.
Comment 24 Egmont Koblinger 2014-06-13 08:30:46 UTC
(In reply to comment #23)
> Ok I have been doing more testings after moving the machine I was ssh'ing into
> from sparc64 to amd64 and it looks like the problem is gone. So it is possible
> the remote ssh connection window in tmux (sparc64) was sending garbled chars
> while with amd64 it doesn't.

VTE should never crash, not even when it receives garbled chars.

> I will keep testing this for a few days and if it still works without issue, I
> will close this bug.

Please don't.  There is definitely a bug and it's not gone.  It might happen that developers are unable to reproduce and hence goes unfixed for years, yeah, life is not perfect... but even if there's no progress for years, let's keep an open record of the issue.
Comment 25 Antoine Jacoutot 2014-06-13 08:36:59 UTC
> Please don't.  There is definitely a bug and it's not gone.  It might happen
> that developers are unable to reproduce and hence goes unfixed for years, yeah,
> life is not perfect... but even if there's no progress for years, let's keep an
> open record of the issue.

Fair enough. I will confirm anyway whether it is still reproducible for me or not in a few days (earlier if it is).
Comment 26 Jasper Lievisse Adriaanse 2014-07-28 08:31:47 UTC
I can confirm this is still an issue (at least on OpenBSD/amd64) and I was able to reproduce it within a few minutes. Here is a full backtrace of the crash:

gdb /usr/local/libexec/gnome-terminal-server -c gnome-terminal-s.core

kgdb\GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-unknown-openbsd5.6"...
Core was generated by `gnome-terminal-s'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libpthread.so.18.0...done.
Loaded symbols for /usr/lib/libpthread.so.18.0
Loaded symbols for /usr/local/libexec/gnome-terminal-server
Reading symbols from /usr/local/lib/libvte2_90.so.1.0...done.
Loaded symbols for /usr/local/lib/libvte2_90.so.1.0
Reading symbols from /usr/local/lib/libgtk-3.so.1200.0...done.
Loaded symbols for /usr/local/lib/libgtk-3.so.1200.0
Reading symbols from /usr/local/lib/libgdk-3.so.1200.0...done.
Loaded symbols for /usr/local/lib/libgdk-3.so.1200.0
Reading symbols from /usr/local/lib/libpangocairo-1.0.so.3600.0...done.
Loaded symbols for /usr/local/lib/libpangocairo-1.0.so.3600.0
Reading symbols from /usr/local/lib/libpango-1.0.so.3600.0...done.
Loaded symbols for /usr/local/lib/libpango-1.0.so.3600.0
Reading symbols from /usr/local/lib/libgobject-2.0.so.4000.0...done.
Loaded symbols for /usr/local/lib/libgobject-2.0.so.4000.0
Reading symbols from /usr/local/lib/libglib-2.0.so.4000.0...done.
Loaded symbols for /usr/local/lib/libglib-2.0.so.4000.0
Reading symbols from /usr/local/lib/libiconv.so.6.0...done.
Loaded symbols for /usr/local/lib/libiconv.so.6.0
Reading symbols from /usr/local/lib/libpcre.so.3.0...done.
Loaded symbols for /usr/local/lib/libpcre.so.3.0
Reading symbols from /usr/local/lib/libintl.so.6.0...done.
Loaded symbols for /usr/local/lib/libintl.so.6.0
Reading symbols from /usr/local/lib/libffi.so.1.0...done.
Loaded symbols for /usr/local/lib/libffi.so.1.0
Reading symbols from /usr/local/lib/libgmodule-2.0.so.4000.0...done.
Loaded symbols for /usr/local/lib/libgmodule-2.0.so.4000.0
Reading symbols from /usr/local/lib/libgthread-2.0.so.4000.0...done.
Loaded symbols for /usr/local/lib/libgthread-2.0.so.4000.0
Reading symbols from /usr/lib/libm.so.9.0...done.
Loaded symbols for /usr/lib/libm.so.9.0
Reading symbols from /usr/local/lib/libcairo.so.12.2...done.
Loaded symbols for /usr/local/lib/libcairo.so.12.2
Symbols already loaded for /usr/lib/libpthread.so.18.0
Reading symbols from /usr/X11R6/lib/libpixman-1.so.32.6...done.
Loaded symbols for /usr/X11R6/lib/libpixman-1.so.32.6
Reading symbols from /usr/X11R6/lib/libpthread-stubs.so.2.0...done.
Loaded symbols for /usr/X11R6/lib/libpthread-stubs.so.2.0
Reading symbols from /usr/X11R6/lib/libfontconfig.so.9.1...done.
Loaded symbols for /usr/X11R6/lib/libfontconfig.so.9.1
Reading symbols from /usr/X11R6/lib/libfreetype.so.22.0...done.
Loaded symbols for /usr/X11R6/lib/libfreetype.so.22.0
Reading symbols from /usr/lib/libz.so.5.0...done.
Loaded symbols for /usr/lib/libz.so.5.0
Reading symbols from /usr/lib/libexpat.so.11.0...done.
Loaded symbols for /usr/lib/libexpat.so.11.0
Reading symbols from /usr/local/lib/libpng.so.17.2...done.
Loaded symbols for /usr/local/lib/libpng.so.17.2
Reading symbols from /usr/X11R6/lib/libxcb-shm.so.1.1...done.
Loaded symbols for /usr/X11R6/lib/libxcb-shm.so.1.1
Reading symbols from /usr/X11R6/lib/libxcb.so.3.1...done.
Loaded symbols for /usr/X11R6/lib/libxcb.so.3.1
Reading symbols from /usr/X11R6/lib/libxcb-render.so.1.0...done.
Loaded symbols for /usr/X11R6/lib/libxcb-render.so.1.0
Reading symbols from /usr/X11R6/lib/libXrender.so.6.0...done.
Loaded symbols for /usr/X11R6/lib/libXrender.so.6.0
Reading symbols from /usr/X11R6/lib/libX11.so.16.0...done.
Loaded symbols for /usr/X11R6/lib/libX11.so.16.0
Reading symbols from /usr/X11R6/lib/libXext.so.13.0...done.
Loaded symbols for /usr/X11R6/lib/libXext.so.13.0
Reading symbols from /usr/local/lib/libpangoft2-1.0.so.3600.0...done.
Loaded symbols for /usr/local/lib/libpangoft2-1.0.so.3600.0
Reading symbols from /usr/local/lib/libharfbuzz.so.4.4...done.
Loaded symbols for /usr/local/lib/libharfbuzz.so.4.4
Reading symbols from /usr/local/lib/libgraphite2.so.0.0...done.
Loaded symbols for /usr/local/lib/libgraphite2.so.0.0
Reading symbols from /usr/local/lib/libgdk_pixbuf-2.0.so.3000.0...done.
Loaded symbols for /usr/local/lib/libgdk_pixbuf-2.0.so.3000.0
Reading symbols from /usr/local/lib/libgio-2.0.so.4000.0...done.
Loaded symbols for /usr/local/lib/libgio-2.0.so.4000.0
Reading symbols from /usr/local/lib/libcairo-gobject.so.1.1...done.
Loaded symbols for /usr/local/lib/libcairo-gobject.so.1.1
Reading symbols from /usr/X11R6/lib/libXinerama.so.6.0...done.
Loaded symbols for /usr/X11R6/lib/libXinerama.so.6.0
Reading symbols from /usr/X11R6/lib/libXi.so.12.1...done.
Loaded symbols for /usr/X11R6/lib/libXi.so.12.1
Reading symbols from /usr/X11R6/lib/libXrandr.so.7.0...done.
Loaded symbols for /usr/X11R6/lib/libXrandr.so.7.0
Reading symbols from /usr/X11R6/lib/libXcursor.so.5.0...done.
Loaded symbols for /usr/X11R6/lib/libXcursor.so.5.0
Reading symbols from /usr/X11R6/lib/libXfixes.so.6.0...done.
Loaded symbols for /usr/X11R6/lib/libXfixes.so.6.0
Reading symbols from /usr/X11R6/lib/libXcomposite.so.4.0...done.
Loaded symbols for /usr/X11R6/lib/libXcomposite.so.4.0
Reading symbols from /usr/X11R6/lib/libXdamage.so.4.0...done.
Loaded symbols for /usr/X11R6/lib/libXdamage.so.4.0
Reading symbols from /usr/local/lib/libatk-1.0.so.21209.1...done.
Loaded symbols for /usr/local/lib/libatk-1.0.so.21209.1
Reading symbols from /usr/local/lib/libatk-bridge-2.0.so.0.0...done.
Loaded symbols for /usr/local/lib/libatk-bridge-2.0.so.0.0
Reading symbols from /usr/local/lib/libdbus-1.so.11.0...done.
Loaded symbols for /usr/local/lib/libdbus-1.so.11.0
Reading symbols from /usr/local/lib/libatspi.so.0.1...done.
Loaded symbols for /usr/local/lib/libatspi.so.0.1
Reading symbols from /usr/X11R6/lib/libSM.so.9.0...done.
Loaded symbols for /usr/X11R6/lib/libSM.so.9.0
Reading symbols from /usr/X11R6/lib/libICE.so.10.0...done.
Loaded symbols for /usr/X11R6/lib/libICE.so.10.0
Reading symbols from /usr/lib/libncurses.so.14.0...done.
Loaded symbols for /usr/lib/libncurses.so.14.0
Reading symbols from /usr/lib/libutil.so.12.1...done.
Loaded symbols for /usr/lib/libutil.so.12.1
Reading symbols from /usr/local/lib/libdconf.so.1.0...done.
Loaded symbols for /usr/local/lib/libdconf.so.1.0
Reading symbols from /usr/local/lib/libuuid.so.14.0...done.
Loaded symbols for /usr/local/lib/libuuid.so.14.0
Reading symbols from /usr/lib/libc.so.77.0...done.
Loaded symbols for /usr/lib/libc.so.77.0
Reading symbols from /usr/X11R6/lib/libXau.so.10.0...done.
Loaded symbols for /usr/X11R6/lib/libXau.so.10.0
Reading symbols from /usr/X11R6/lib/libXdmcp.so.11.0...done.
Loaded symbols for /usr/X11R6/lib/libXdmcp.so.11.0
Reading symbols from /usr/lib/libstdc++.so.57.0...done.
Loaded symbols for /usr/lib/libstdc++.so.57.0
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
Reading symbols from /usr/local/lib/gio/modules/libgvfsdbus.so...done.
Loaded symbols for /usr/local/lib/gio/modules/libgvfsdbus.so
Reading symbols from /usr/local/lib/gvfs/libgvfscommon.so...done.
Loaded symbols for /usr/local/lib/gvfs/libgvfscommon.so
Reading symbols from /usr/local/lib/gtk-3.0/3.0.0/theming-engines/libadwaita.so...done.
Loaded symbols for /usr/local/lib/gtk-3.0/3.0.0/theming-engines/libadwaita.so
Reading symbols from /usr/local/lib/librsvg-2.so.38.1...done.
Loaded symbols for /usr/local/lib/librsvg-2.so.38.1
Reading symbols from /usr/local/lib/libcroco-0.6.so.4.0...done.
Loaded symbols for /usr/local/lib/libcroco-0.6.so.4.0
Reading symbols from /usr/local/lib/libxml2.so.15.1...done.
Loaded symbols for /usr/local/lib/libxml2.so.15.1
Reading symbols from /usr/local/lib/gtk-3.0/modules/libcanberra-gtk-module.so...done.
Loaded symbols for /usr/local/lib/gtk-3.0/modules/libcanberra-gtk-module.so
Reading symbols from /usr/local/lib/libcanberra.so.2.0...done.
Loaded symbols for /usr/local/lib/libcanberra.so.2.0
Reading symbols from /usr/local/lib/libvorbisfile.so.6.0...done.
Loaded symbols for /usr/local/lib/libvorbisfile.so.6.0
Reading symbols from /usr/local/lib/libvorbis.so.8.0...done.
Loaded symbols for /usr/local/lib/libvorbis.so.8.0
Reading symbols from /usr/local/lib/libogg.so.6.2...done.
Loaded symbols for /usr/local/lib/libogg.so.6.2
Reading symbols from /usr/local/lib/libltdl.so.5.0...done.
Loaded symbols for /usr/local/lib/libltdl.so.5.0
Reading symbols from /usr/local/lib/libcanberra-gtk3.so.1.8...done.
Loaded symbols for /usr/local/lib/libcanberra-gtk3.so.1.8
Reading symbols from /usr/local/lib/gio/modules/libdconfsettings.so...done.
Loaded symbols for /usr/local/lib/gio/modules/libdconfsettings.so
Reading symbols from /usr/local/lib/pango/1.8.0/modules/pango-basic-fc.so...done.
Loaded symbols for /usr/local/lib/pango/1.8.0/modules/pango-basic-fc.so
Reading symbols from /usr/local/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so...done.
Loaded symbols for /usr/local/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so
  • #0 memcpy
    at /usr/src/lib/libc/string/bcopy.c line 115
  • #0 memcpy
    at /usr/src/lib/libc/string/bcopy.c line 115
  • #1 vte_terminal_accessible_get_text
    at vteaccess.c line 1075
  • #2 text_changed_event_listener
    at event.c line 892
  • #3 signal_emit_unlocked_R
    at gsignal.c line 3517
  • #4 g_signal_emit_valist
    at gsignal.c line 3307
  • #5 g_signal_emit_by_name
    at gsignal.c line 3403
  • #6 emit_text_changed_delete
    at vteaccess.c line 234
  • #7 vte_terminal_accessible_text_modified
    at vteaccess.c line 513
  • #8 g_cclosure_marshal_VOID__VOIDv
    at gmarshal.c line 115
  • #9 _g_closure_invoke_va
    at gclosure.c line 831
  • #10 g_signal_emit_valist
    at gsignal.c line 3215
  • #11 g_signal_emit_by_name
    at gsignal.c line 3403
  • #12 _vte_terminal_emit_text_inserted
    at vte.c line 971
  • #13 vte_terminal_emit_pending_text_signals
    at vte.c line 4104
  • #14 vte_terminal_emit_pending_signals
    at vte.c line 14734
  • #15 vte_terminal_process_incoming
    at vte.c line 4488
  • #16 time_process_incoming
    at vte.c line 14745
  • #17 process_timeout
    at vte.c line 14792
  • #18 g_timeout_dispatch
    at gmain.c line 4472
  • #19 g_main_dispatch
    at gmain.c line 3064
  • #20 g_main_context_dispatch
    at gmain.c line 3663
  • #21 g_main_context_iterate
    at gmain.c line 3734
  • #22 g_main_context_iteration
    at gmain.c line 3795
  • #23 g_application_run
    at gapplication.c line 2115
  • #24 main
    at server.c line 144
  • #0 memcpy
    at /usr/src/lib/libc/string/bcopy.c line 115
  • #1 vte_terminal_accessible_get_text
    at vteaccess.c line 1075
  • #2 text_changed_event_listener
    at event.c line 892
  • #3 signal_emit_unlocked_R
    at gsignal.c line 3517
  • #4 g_signal_emit_valist
    at gsignal.c line 3307
  • #5 g_signal_emit_by_name
    at gsignal.c line 3403
  • #6 emit_text_changed_delete
    at vteaccess.c line 234
  • #7 vte_terminal_accessible_text_modified
    at vteaccess.c line 513
  • #8 g_cclosure_marshal_VOID__VOIDv
    at gmarshal.c line 115
  • #9 _g_closure_invoke_va
    at gclosure.c line 831
  • #10 g_signal_emit_valist
    at gsignal.c line 3215
  • #11 g_signal_emit_by_name
    at gsignal.c line 3403
  • #12 _vte_terminal_emit_text_inserted
    at vte.c line 971
  • #13 vte_terminal_emit_pending_text_signals
    at vte.c line 4104
  • #14 vte_terminal_emit_pending_signals
    at vte.c line 14734
  • #15 vte_terminal_process_incoming
    at vte.c line 4488
  • #16 time_process_incoming
    at vte.c line 14745
  • #17 process_timeout
    at vte.c line 14792
  • #18 g_timeout_dispatch
    at gmain.c line 4472
  • #19 g_main_dispatch
    at gmain.c line 3064
  • #20 g_main_context_dispatch
    at gmain.c line 3663
  • #21 g_main_context_iterate
    at gmain.c line 3734
  • #22 g_main_context_iteration
    at gmain.c line 3795
  • #23 g_application_run
    at gapplication.c line 2115
  • #24 main
    at server.c line 144

Comment 27 Egmont Koblinger 2014-07-28 09:12:17 UTC
Jasper and Antoine, could you please help by:

- Trying to find a way to trigger the crash as quickly and as reliably as possible?  (Ideally, a way that always makes vte crash by the same few steps would be the best.)

- Grant ssh access to an openbsd box with standard development tools installed?  (If you can, let's discuss the details in private mail.)
Comment 28 Antoine Jacoutot 2014-07-28 15:38:20 UTC
Hi Egmont.

As mentioned in the very post, it is not 100% reproducible so there are no steps really. As already mentioned, having a tmux window hosting the calcurse software in a window and something else in another window and going back and forth between the two is how I am able to reproduce it most of the times.
But well, it also happened twice already that gnome-terminal-server crashed displaying the output of a software compilation.

Granting ssh access to you should be no problem but it seems that for now, I am only able to reproduce this issue on accelerated display; which is not possible to check remotely...
Comment 29 Jasper Lievisse Adriaanse 2014-07-28 17:08:05 UTC
(In reply to comment #28)
> Hi Egmont.
> 
> As mentioned in the very post, it is not 100% reproducible so there are no
> steps really. As already mentioned, having a tmux window hosting the calcurse
> software in a window and something else in another window and going back and
> forth between the two is how I am able to reproduce it most of the times.
> But well, it also happened twice already that gnome-terminal-server crashed
> displaying the output of a software compilation.

This is the fastest way for me to reproduce it too, open calcurse, add an appointment with UTF-8 characters then cycle between tmux windows, attaching/detaching it between various gnome-terminal windows every 10 or so times. That way I could force a crash in under two minutes.
Comment 30 Robert Nagy 2014-07-28 21:18:05 UTC
Created attachment 281888 [details]
print utf8 chars randomly
Comment 31 Robert Nagy 2014-07-28 21:18:31 UTC
Hello,

I've created a simple test case for you guys to try to reproduce it.

I've attached a symple ruby script that prints UTF-8 characters (Hungarian) onto the terminal.

1. run tmux and start the script in one window
2. open a new empty window in tmux and start the following loop.
   - while true ; do tmux last ; sleep 0.2 ; done 

This will automatically switch between the two windows in tmux and will trigger the crash in about 10-15 minutes.
Comment 32 Antoine Jacoutot 2014-07-29 14:35:23 UTC
Created attachment 281956 [details]
use sizeof v2

Ok, turned out my initial patch was almost correct. Here's a fixed one from Robert Nagy. I haven't been able to crash gnome-terminal for the last 2 hours running the crash-test (it usually crashes under 60 seconds).
start and end are integers while memcpy() expects a size_t.

Thoughts?
Comment 33 Jasper Lievisse Adriaanse 2014-07-29 14:55:58 UTC
I've not been able to crash gnome-terminal since starting the "torture" script several hours ago. Rationale and patch seem correct to me.
Comment 34 Egmont Koblinger 2014-07-30 08:56:44 UTC
Comment on attachment 281956 [details]
use sizeof v2

Just as bad as the previous patch for the exact same reason (comments 5 & 16). You misunderstand sizeof() and use it fundamentally incorrectly.
Comment 35 Antoine Jacoutot 2014-07-30 08:59:36 UTC
(In reply to comment #34)
> (From update of attachment 281956 [details])
> Just as bad as the previous patch for the exact same reason (comments 5 & 16).
> You misunderstand sizeof() and use it fundamentally incorrectly.

Yes sorry about that; we are actually using another patch.
Comment 36 Antoine Jacoutot 2014-07-30 09:00:06 UTC
Created attachment 282004 [details] [review]
clamp the end computation to the string  length
Comment 37 Egmont Koblinger 2014-07-30 09:08:00 UTC
Thanks. Looks reasonable for a quick workaround, but I'd like to further investigate. How come 'end' is too large sometimes, why is this? What's the contents of str then, is it correct or garbled? I feel the patch is probably fixing the symptom rather than the real bug.
Comment 38 Antoine Jacoutot 2014-07-30 09:32:09 UTC
(In reply to comment #37)
> Thanks. Looks reasonable for a quick workaround, but I'd like to further
> investigate. How come 'end' is too large sometimes, why is this? What's the
> contents of str then, is it correct or garbled? I feel the patch is probably
> fixing the symptom rather than the real bug.

Correct, we are fixing the symptom here.

str looks like this for me:

current = 0x8034d000 "�\224\214�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224\200�\224"...
Comment 39 Egmont Koblinger 2014-07-31 23:08:49 UTC
I installed OpenBSD under VMware, and could successfully reproduce the crash.

Boring details:
- I'm using fvwm (gnome doesn't start due to missing hardware graphic acceleration);
- running LANG=C.UTF-8 ./src/vte2_90 (i.e. vteapp 0.36; 0.37 doesn't compile due to glib dependency), half-maximized window (whatever fvwm's maximize button does);
- the test case from comment 31, but with sleep 0.1 instead of 0.2 (this seems to be important);
- starting up "orca" previously, and I've also enabled some a11y stuff in gconf, not sure how much they're required;
- crashes in like 10-15 seconds (not minutes) which is great;
- couldn't crash on Linux, but I got a couple of "atk-bridge: Received bad UTF-8 string when emitting event" warnings which might be the same bug.
Comment 40 Antoine Jacoutot 2014-08-01 06:19:29 UTC
(In reply to comment #39)
> I installed OpenBSD under VMware, and could successfully reproduce the crash.

Awesome! (well you know :-)).

> Boring details:
> - I'm using fvwm (gnome doesn't start due to missing hardware graphic
> acceleration);

Right, you would need to compile the llvmpipe drivers; they are not compiled by default because they require clang/llvm which is not in base yet.

> - couldn't crash on Linux, but I got a couple of "atk-bridge: Received bad
> UTF-8 string when emitting event" warnings which might be the same bug.

Interesting. Thanks Egmont.
Comment 41 Egmont Koblinger 2014-08-01 08:42:21 UTC
Okay, I think I got it :)

vte_terminal_accessible_get_text() relies on both priv->snapshot_text and priv->snapshot_characters being properly set to return the desired text.

When deleting text, this function is called [from emit_text_changed_delete() via signals] with the start and end offsets according to the *old* text, even though the accessibility snapshot was already updated to the new text.

Having the old values there is achieved by temporarily reverting and then restoring priv->snapshot_text just before/after calling emit_text_changed_delete() -- however, this code forgets to do the same with priv->snapshot_characters.
Comment 42 Egmont Koblinger 2014-08-01 09:51:27 UTC
Created attachment 282245 [details] [review]
Fix (for vte 0.36)

Here's a fix.  Could you guys please test this and let me know if it works for you too?

I did a bit of cleanup wrt returning the old snapshot_text, i.e. return the GString instead of str and len separately.  My goal was to make this and snapshot_characters (GArray) quite alike.  [Coincidentally, most of the calls to vte_terminal_accessible_update_private_data_if_needed() didn't need to be updated because they still pass two NULLs, although their meaning has changed.]  I'm not completely there yet.  I still need to study memory management of glib a bit, to see if I can use ref() on the string instead of copying, and whether totally unrefing a GArray or GString to zero refcount calls g_whatever_free with a FALSE or TRUE second argument, to make sure I don't leak.
Comment 43 Antoine Jacoutot 2014-08-01 10:06:29 UTC
(In reply to comment #42)
> Created an attachment (id=282245) [details] [review]
> Fix (for vte 0.36)
> 
> Here's a fix.  Could you guys please test this and let me know if it works for
> you too?

So far it seems to do the trick \o/
I will let the test run for a few and see if that also fixes my regular issues on my laptop. If you don't hear from me soon, then it means everything works :-)

Thanks.
Comment 44 Egmont Koblinger 2014-08-01 22:13:15 UTC
Created attachment 282296 [details] [review]
Fix v2 (for vte 0.36)

Could you guys please update to this second patch?  This is the one I'm planning to commit, unless someone objects.

I figured out I didn't understand the code enough to safely use g_array_ref and it's fragile against further changes (there are two parts of the code that expect the other not to modify the data), so I went for copying instead.  This makes the code more robust and more readable too because handling snapshot_text and snapshot_characters are now more similar to each other.

Plus minor indentation changes.
Comment 45 Egmont Koblinger 2014-08-01 22:29:52 UTC
Created attachment 282297 [details] [review]
Fix v2 (for vte 0.37)
Comment 46 Antoine Jacoutot 2014-08-02 07:58:16 UTC
Hi Egmont. Just so you know, I will not be able to test this for real before Wednesday... :-/
Jasper/Robert if you could test, it'd be nice, so that we have this fix committed soon.
Thanks.
Comment 47 Egmont Koblinger 2014-08-02 16:21:23 UTC
Created attachment 282342 [details] [review]
Fix v3 (for vte 0.36)
Comment 48 Egmont Koblinger 2014-08-02 16:21:52 UTC
Created attachment 282343 [details] [review]
Fix v3 (for vte 0.37)
Comment 49 Jasper Lievisse Adriaanse 2014-08-04 12:58:14 UTC
Hi Egmont, v3 (for 0.36) has been holding up just here running the test for a few hours. Thank you!
Comment 50 Egmont Koblinger 2014-08-04 13:14:29 UTC
Comment on attachment 282342 [details] [review]
Fix v3 (for vte 0.36)

committed to vte-0-36 as 4664d7c
Comment 51 Egmont Koblinger 2014-08-04 13:14:50 UTC
Comment on attachment 282343 [details] [review]
Fix v3 (for vte 0.37)

committed to master as a20f7d2
Comment 52 Egmont Koblinger 2014-08-04 13:35:06 UTC
You're welcome :)

The patches are now in the git repo.  I can't tell if we'll make a 0.36.4 release, depends on the main developer who's away now.

As for OpenBSD it seems to me that you guys are developers there so maybe you can cherrypick the fix and release an update?  Up to you.

Let me mention that the bug lies in the accessibility part of vte.  For me (on Ubuntu) that code is not run at all, unless I enable Universal Access -> Screen Reader in Gnome's settings.  Maybe you guys all enabled it for whatever reason (that's fine and none of my business), but maybe OpenBSD's Gnome enables it by default for everyone, in that case this is something that should probably be changed in OpenBSD.
Comment 53 Antoine Jacoutot 2014-08-04 14:54:42 UTC
> As for OpenBSD it seems to me that you guys are developers there so maybe you
> can cherrypick the fix and release an update?  Up to you.

Hi Egmont.

Yeah that's correct, we will integrate this patch in the OpenBSD vte package.

> Let me mention that the bug lies in the accessibility part of vte.  For me (on
> Ubuntu) that code is not run at all, unless I enable Universal Access -> Screen
> Reader in Gnome's settings.  Maybe you guys all enabled it for whatever reason
> (that's fine and none of my business), but maybe OpenBSD's Gnome enables it by
> default for everyone, in that case this is something that should probably be
> changed in OpenBSD.

That is odd. None of the accessibility features are enabled by default on OpenBSD. And I just checked my 'Screen Reader' accessibility config and it is set to 'Off' ... Not sure how we end up running that code while it's not for you...
Comment 54 Egmont Koblinger 2014-08-04 16:06:50 UTC
I don't know where the decision is made whether to enable a11y.  I didn't find relevant code in vte.  I guess it's hidden somewhere in atk/gtk.  Unfortunately I don't know any more about this story.  Could be worth it filing a bug against them.