From mdo1@dbatest.cit.cornell.edu Fri Mar 28 15:50:40 1997
Received: from MIT.EDU (PACIFIC-CARRIER-ANNEX.MIT.EDU [18.69.0.28]) by rt-11.MIT.EDU (8.7.5/8.7.3) with SMTP id PAA10689 for <bugs@RT-11.MIT.EDU>; Fri, 28 Mar 1997 15:50:40 -0500
Received: from MIT.MIT.EDU by MIT.EDU with SMTP
id AA12248; Fri, 28 Mar 97 15:50:26 EST
Received: from DBATEST.CIT.CORNELL.EDU by MIT.MIT.EDU (5.61/4.7) id AA28978; Fri, 28 Mar 97 15:50:38 EST
Received: by dbatest.cit.cornell.edu (AIX 4.1/UCB 5.64/4.03)
id AA07360; Fri, 28 Mar 1997 15:49:42 -0500
Message-Id: <9703282049.AA07360@dbatest.cit.cornell.edu>
Date: Fri, 28 Mar 1997 15:49:42 -0500
From: mdo1@dbatest.cit.cornell.edu (Michael D Oltz)
Reply-To: mdo1@dbatest.cit.cornell.edu
To: krb5-bugs@MIT.EDU
Subject: AIX 4.x pty death
X-Send-Pr-Version: 3.99
Show quoted text
>Number: 406
>Category: pty
>Synopsis: AIX 4.x pty death
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: hartmans
>State: closed
>Class: sw-bug
>Submitter-Id: unknown
>Arrival-Date: Fri Mar 28 15:51:01 EST 1997
>Last-Modified: Thu Sep 13 23:27:03 EDT 2001
>Originator: Michael D Oltz
>Organization:
Cornell University
Show quoted text
>Release: 1.0
>Environment:
RS/6000, AIX 4.1.4, target same, built with bundled compiler,
--with-krb4 but not with shared libraries
System: AIX dbatest 1 4 000057973500
Show quoted text
>Description:
tried to use the 16-bit NCSA Telnet for Windows bundled
with the K5 1.0 release to connect to my K5 server host
running the K5 telnetd (on a high port number as an
alternate). daemon gets through authentication okay,
then it calls startslave. when it gets to the fork
call, the daemon causes a hard crash (cold boot required).
i haven't stepped into the kernel yet, but it crashes
fast enough that gdb doesn't report the creation of a
new thread. it could be that the child starts, but dies
before the parent gets to return from the fork()
p.s. i can gssftp from aforementioned
host to itself without trouble. but self-telnet dies.
connecting with any vanilla telnet client causes the
same problem. any advice? anybody else had this problem?
Show quoted text
>How-To-Repeat:
see above. well, that's what happens to me.
Show quoted text
>Fix:
Show quoted text
>Audit-Trail:
From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
To: krb5-bugs@MIT.EDU, mdo1@dbatest.cit.cornell.edu
Cc: krb5-unassigned@RT-11.MIT.EDU, gnats-admin@RT-11.MIT.EDU,
krb5-prs@RT-11.MIT.EDU
Subject: Re: krb5-appl/406: telnetd hoses the host at fork()
Date: Mon, 31 Mar 1997 19:21:41 -0500
Date: Fri, 28 Mar 1997 15:49:42 -0500
From: mdo1@dbatest.cit.cornell.edu (Michael D Oltz)
tried to use the 16-bit NCSA Telnet for Windows bundled
with the K5 1.0 release to connect to my K5 server host
running the K5 telnetd (on a high port number as an
alternate). daemon gets through authentication okay,
then it calls startslave. when it gets to the fork
call, the daemon causes a hard crash (cold boot required).
i haven't stepped into the kernel yet, but it crashes
fast enough that gdb doesn't report the creation of a
new thread. it could be that the child starts, but dies
before the parent gets to return from the fork()
p.s. i can gssftp from aforementioned
host to itself without trouble. but self-telnet dies.
connecting with any vanilla telnet client causes the
same problem. any advice? anybody else had this problem?
This is a known problem with AIX 4.1; our pty handling is somehow
causing AIX 4.x to lock up and crash. While this clearly means there's
a bug in our pty library, it's also a major bug in AIX that a user
program can cause the kernel to panic.
We don't have any AIX machines within easy access of our development
team, so we're going to be relying on others (who have the misfortune of
having to use AIX :-) to try to fix this problem. For whatever it's
worth, our pty code works just fine under AIX 3.x; not only do the
kernels not panic, but telnetd and rlogind actually work! Clearly, IBM
changed some very significant things in its pty handling code between
AIX 3 and AIX 4.
If anyone has any suggestions, or better yet patches, we would be
greatly appreciative of them.
- Ted
Responsible-Changed-From-To: krb5-unassigned->hartmans
Responsible-Changed-By: tytso
Responsible-Changed-When: Mon Mar 31 19:23:06 1997
Responsible-Changed-Why: Moved to category "pty"
From: Doug Engert <DEEngert@anl.gov>
To: "Theodore Y. Ts'o" <tytso@MIT.EDU>
Cc: krb5-bugs@MIT.EDU, mdo1@dbatest.cit.cornell.edu,
krb5-unassigned@RT-11.MIT.EDU, gnats-admin@RT-11.MIT.EDU,
krb5-prs@RT-11.MIT.EDU
Subject: Re: krb5-appl/406: telnetd hoses the host at fork()
Date: Tue, 1 Apr 1997 09:18:15 -0600
Theodore Y. Ts'o writes:
Show quoted text
> Date: Fri, 28 Mar 1997 15:49:42 -0500
> From: mdo1@dbatest.cit.cornell.edu (Michael D Oltz)
>
> tried to use the 16-bit NCSA Telnet for Windows bundled
> with the K5 1.0 release to connect to my K5 server host
> running the K5 telnetd (on a high port number as an
> alternate). daemon gets through authentication okay,
> then it calls startslave. when it gets to the fork
> call, the daemon causes a hard crash (cold boot required).
> i haven't stepped into the kernel yet, but it crashes
> fast enough that gdb doesn't report the creation of a
> new thread. it could be that the child starts, but dies
> before the parent gets to return from the fork()
> p.s. i can gssftp from aforementioned
> host to itself without trouble. but self-telnet dies.
> connecting with any vanilla telnet client causes the
> same problem. any advice? anybody else had this problem?
Yes. login.krb5 hangs my AIX 4.1.4 systems as well. I have not tried
it on a AIX 4.2 system yet.
A circumvention is to use the AIX login instead of the login.krb5.
This will work for the telnetd and rlogind. But not for a login at the
console. Have sbin/login.krb5 -> /usr/bin/login. I am doing this on our
AIX 4.1.4 and AIX 4.2 systems. Both are using the K5-1.0 built on AIX
4.1.4 (I actually have login.krb5 -> k5prelogin which checks for DCE
and/or AFS and execs k5dcelogin and/or k5afslogin, which then exec
/usr/bin/login instead of the login.krb5)
Show quoted text
>
> This is a known problem with AIX 4.1; our pty handling is somehow
> causing AIX 4.x to lock up and crash. While this clearly means there's
> a bug in our pty library, it's also a major bug in AIX that a user
> program can cause the kernel to panic.
>
> We don't have any AIX machines within easy access of our development
> team, so we're going to be relying on others (who have the misfortune of
> having to use AIX :-) to try to fix this problem. For whatever it's
> worth, our pty code works just fine under AIX 3.x; not only do the
> kernels not panic, but telnetd and rlogind actually work! Clearly, IBM
> changed some very significant things in its pty handling code between
> AIX 3 and AIX 4.
>
> If anyone has any suggestions, or better yet patches, we would be
> greatly appreciative of them.
>
> - Ted
I noticed today that the configure.in in the util/pty directory has the
following:
*-*-aix3*) # AIX has streams include files but not streams TTY
# Moreover, strops.h trashes sys/ioctl.h
krb5_cv_has_streams=no
Is this the same problem, or a similar problem: mixing streams pty
calls with a non-stream driver? I can't find a strops.h but there is a
sys/stropts.h on AIX 4.1.4 which does get included by
util/pty/pty-int.h Yet the device which appears to get opened is the
/dev/ptc which I don't believe is the streams driver.
Should the test be expanded to include AIX 4?
Try removing the -DHAVE_STREAMS=1 from the util/pty/Makefile and
rebuilding the lib, and telnetd, and see if it still fails.
--
Douglas E. Engert <DEEngert@anl.gov>
Argonne National Laboratory
9700 South Cass Avenue
Argonne, Illinois 60439
(630) 252-5444
From: Michael Oltz <mdo1@cornell.edu>
To: Doug Engert <DEEngert@anl.gov>
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-appl/406 and telnet/417
Date: Tue, 15 Apr 1997 16:14:46 -0400
Show quoted text
>A circumvention is to use the AIX login instead of the login.krb5.
>This will work for the telnetd and rlogind. But not for a login at the
>console. Have sbin/login.krb5 -> /usr/bin/login. I am doing this on our
>AIX 4.1.4 and AIX 4.2 systems. Both are using the K5-1.0 built on AIX
>4.1.4 (I actually have login.krb5 -> k5prelogin which checks for DCE
>and/or AFS and execs k5dcelogin and/or k5afslogin, which then exec
>/usr/bin/login instead of the login.krb5)
On the machine I am using as a "sandbox", which was inherited from
someone else, /usr/bin/login is itself a symbolic link to
/usr/sbin/trm . There is also a /usr/sbin/login, but it's binary
identical to /usr/sbin/trm. I symbolic-linked sbin/login.krb5 to
/usr/sbin/trm, and what I get instead in the debug output sent
to the client is this:
[after much non-problematic-looking verbiage:]
4.4 BSD UNIX (dbatest) (1)
td: netflush 113 chars
ogin: Not a td: netflush 12 chars
ecognized flag: F
td: netflush 21 chars
td: netflush 43 chars
dev/pts/1: 3004-023 -"F" is not a valid option to login.
After which the client and the daemon quietly terminate (i.e. without
a system crash).
Show quoted text
>Try removing the -DHAVE_STREAMS=1 from the util/pty/Makefile and
>rebuilding the lib, and telnetd, and see if it still fails.
I tried this too, and now the client and the daemon both quietly
and rather quickly terminate before they get to the herald line shown
above. But this is vastly preferable to crashing the machine.
From: Tom Yu <tlyu@MIT.EDU>
To: Sam Hartman <hartmans@MIT.EDU>
Cc: krb5-bugs@MIT.EDU
Subject: Re: pty/406: AIX 4.x pty death
Date: Sun, 22 Feb 1998 21:19:07 -0500
`Tom Yu' made changes to this PR.
--- /tmp/gnatsa003n_ Sun Feb 22 21:18:46 1998
+++ /tmp/gnatsb003n_ Sun Feb 22 21:19:05 1998
@@ -10,12 +10,12 @@
From: mdo1@dbatest.cit.cornell.edu (Michael D Oltz)
Reply-To: mdo1@dbatest.cit.cornell.edu
To: krb5-bugs@MIT.EDU
-Subject: telnetd hoses the host at fork()
+Subject: AIX 4.x pty death
X-Send-Pr-Version: 3.99
Show quoted text
>Number: 406
>Category: pty
->Synopsis: telnetd hoses the host at fork()
+>Synopsis: AIX 4.x pty death
Show quoted text
>Confidential: no
>Severity: critical
>Priority: high
State-Changed-From-To: open-closed
State-Changed-By: tlyu
State-Changed-When: Thu Sep 13 23:26:42 2001
State-Changed-Why:
believed to be irrelevant in modern releases of AIX
Show quoted text
>Unformatted:
