Distribution/Version: 10

We are running vte 0.11.11 on Solaris 10.  When a gnome-terminal is started,
two ptys are apparently allocated -- one is the actual controlling terminal
for the shell process, while another is recorded in the corresponding utmp
record.  It seems that the problem is in the build system -- gnome-pty-helper
is configured to use sendmsg() to pass the pty descriptors to the vte client,
but the client code (in src/pty.c) is not configured to use recvmsg().  The
client falls back to allocating a pty itself.

The problem can be fixed by correcting the autoconf test for recvmsg(), which
is found in the socket library on Solaris.  (gnome-pty-helper already has
a proper test for sendmsg()).  The following patch (against our local cvs
repository) to the top-level configure.in (and regenerating configure) fixes
the problem.

Index: configure.in
===================================================================
RCS file: /afs/dev.mit.edu/source/repository/third/vte/configure.in,v
retrieving revision 1.4
diff -u -r1.4 configure.in
--- configure.in        22 Dec 2004 17:27:27 -0000      1.4
+++ configure.in        19 May 2005 21:48:36 -0000
@@ -225,17 +225,21 @@
 AC_HEADER_TIOCGWINSZ
 
 # Check for PTY handling functions.
-AC_CHECK_FUNCS(cfmakeraw getpgid getpt grantpt unlockpt ptsname ptsname_r recvmsg)
+AC_CHECK_FUNCS(cfmakeraw getpgid getpt grantpt unlockpt ptsname ptsname_r)
 
 # Pull in the right libraries for various functions which might not be
 # bundled into an exploded libc.
 AC_CHECK_FUNC(socket,[have_socket=1],AC_CHECK_LIB(socket,socket,[have_socket=1;
LIBS="$LIBS -lsocket"]))
 AC_CHECK_FUNC(socketpair,[have_socketpair=1],AC_CHECK_LIB(socket,socketpair,[have_socketpair=1;
LIBS="$LIBS -lsocket"]))
+AC_CHECK_FUNC(recvmsg,[have_recvmsg=1],AC_CHECK_LIB(socket,recvmsg,[have_recvmsg=1;
LIBS="$LIBS -lsocket -lnsl"]))
 if test x$have_socket = x1 ; then
        AC_DEFINE(HAVE_SOCKET,1,[Define if you have the socket function.])
 fi
 if test x$have_socketpair = x1 ; then
        AC_DEFINE(HAVE_SOCKETPAIR,1,[Define if you have the socketpair function.])
+fi
+if test x$have_recvmsg = x1 ; then
+       AC_DEFINE(HAVE_RECVMSG,1,[Define if you have the recvmsg function.])
 fi
 AC_CHECK_FUNC(floor,,AC_CHECK_LIB(m,floor,LIBS=["$LIBS -lm"]))