Ticket History #873: AIX 4.3 seteuid process can't fork if root has too many procs

From donn@u.washington.edu Fri Aug 4 17:32:59 2000
Received: from fort-point-station.mit.edu (FORT-POINT-STATION.MIT.EDU [18.72.0.53])
by rt-11.mit.edu (8.9.3/8.9.3) with ESMTP id RAA16736
for <bugs@RT-11.MIT.EDU>; Fri, 4 Aug 2000 17:32:58 -0400 (EDT)
Received: from melville.u.washington.edu (donn@melville.u.washington.edu [128.95.135.35])
by fort-point-station.mit.edu (8.9.2/8.9.2) with ESMTP id RAA00799
for <krb5-bugs@mit.edu>; Fri, 4 Aug 2000 17:32:57 -0400 (EDT)
Received: (from donn@localhost)
by melville.u.washington.edu (8.9.3+UW00.05/8.9.3+UW99.09) id OAA61208;
Fri, 4 Aug 2000 14:32:56 -0700
Message-Id: <200008042132.OAA61208@melville.u.washington.edu>
Date: Fri, 4 Aug 2000 14:32:56 -0700
From: donn@u.washington.edu
Reply-To: donn@u.washington.edu
To: krb5-bugs@mit.edu
Subject: AIX 4.3 seteuid broken
X-Send-Pr-Version: 3.99

University Computing Services
University of Washington
AIX 4.3.3 standard stuff.
System: AIX version 4.3.3


AIX 4.3 seteuid clears some privilege bits that would (probably
among other things) allow the process to exceed maximum process limit.
Then when the process attempts to fork, process accounting is apparently
done against the real UID. In something like ftpd, that's 0 (root),
and the processes may very well be over the limit, since of course
root isn't subject to the limit. So ftpd can't spawn an "ls".
We brought this to IBM's attention, but from what I hear they do not
intend to fix it.

Up through AIX 4.2 this was not a problem. I realize this is pure
brain damage, and my fix isn't that beautiful either. But FYI,
here's the problem and here's how I fix it.

Ftp to an AIX host, and send the "ls" or "dir" command. If
1. the ftpd software is MIT Kerberos 5 ftpd or anything like it, and
2. root has more processes than the user process limit, and
3. the OS version is 4.3.3

the dir will fail with the message
"550 /bin/ls: Resource temporarily unavailable."

*** lib/krb5util/seteuid.c.dist Thu Jun 29 19:28:14 2000
--- lib/krb5util/seteuid.c Fri Aug 4 13:40:18 2000
***************
*** 34,39 ****
--- 34,64 ----
#include <stdlib.h>
#endif

+ #ifdef _AIX
+ /*
+ ** AIX 4.3 seteuid clears some privilege bits that would (probably
+ ** among other things) allow the process to exceed maximum process limit.
+ ** Then when the process attempts to fork, process accounting is apparently
+ ** done against the real UID. In something like ftpd, that's 0 (root),
+ ** and the processes may very well be over the limit, since of course
+ ** root isn't subject to the limit. So ftpd can't spawn an "ls".
+ ** We brought this to IBM's attention, but from what I hear they do not
+ ** intend to fix it. To work around this brain damage for ftpd's sake,
+ ** we can either turn the privs back on afterwards (!) or use setuidx().
+ ** The latter looks like the best bet to me, should work fine with older
+ ** versions of AIX if necessary.
+ **
+ ** Donn Cave, University of Washington
+ */
+
+ /* Guess if this is AIX 4.3 */
+ #include <sys/var.h>
+ #ifdef SYSP_GET
+ #define USE_SETUIDX 1
+ #include <sys/id.h>
+ #endif
+ #endif /* AIX */
+
#include <errno.h>

int krb5_seteuid(euid_in)
***************
*** 40,45 ****
--- 65,73 ----
int euid_in;
{
uid_t euid = (uid_t) euid_in;
+ #ifdef USE_SETUIDX
+ return setuidx(ID_EFFECTIVE, euid);
+ #else
#if defined(HAVE_SETEUID)
return (seteuid(euid));
#else
***************
*** 55,60 ****
--- 83,89 ----
#endif /* HAVE_SETREUID */
#endif /* HAVE_SETRESUID */
#endif /* HAVE_SETEUID */
+ #endif /* AIX */
}

int krb5_setegid(egid_in)

Date: Tue, 12 Dec 2000 14:51:14 -0800
From: donn@u.washington.edu

In krb5-libs/873 I reported a problem with AIX 4.3 seteuid(),
which exposes the process to root quota issues that would
normally be immaterial for root.
In the fix, however, I proposed to use setuidx(), and that
turns out to be much too generous with privilege, and I want
to withdraw that. That is my main objective here.
The following is offered as a substitute, though IBM has
indicated they would fix seteuid() in future maintenance
releases. My fix here is to set only the privilege bit that
pertains to the observed problem.
Ftp to host with many root processes, try to "ls" (this shows
the original problem, root's process quota vs. seteuid.
*** seteuid.c.dist Thu Jun 29 19:28:14 2000
--- seteuid.c Tue Dec 12 12:55:52 2000
***************
*** 34,47 ****
#include <stdlib.h>
#endif

#include <errno.h>

int krb5_seteuid(euid_in)
int euid_in;
{
uid_t euid = (uid_t) euid_in;
#if defined(HAVE_SETEUID)
! return (seteuid(euid));
#else
#if defined(HAVE_SETRESUID)
return (setresuid(getuid(), euid, geteuid()));
--- 34,87 ----
#include <stdlib.h>
#endif

+ #ifdef _AIX
+ /*
+ ** AIX 4.3 seteuid clears some privilege bits that would (probably
+ ** among other things) allow the process to exceed maximum process limit.
+ ** Then when the process attempts to fork, process accounting is apparently
+ ** done against the real UID. In something like ftpd, that's 0 (root),
+ ** and the processes may very well be over the limit, since of course
+ ** root isn't subject to the limit. So ftpd can't spawn an "ls".
+ ** We brought this to IBM's attention, and a fix is supposed to be on
+ ** the way, some day. To work around this brain damage for ftpd's sake,
+ ** we can either turn the privs back on afterwards (!) or use setuidx().
+ **
+ ** It turns out that either leaves the potential to create files in
+ ** directories owned by root, which we do not like. setuidx() does
+ ** this, and full priv restoration does it. However, I find that
+ ** I can avoid the quota problem with a single bit, SET_PROC_RAC, so
+ ** that's my work-around.
+ **
+ ** Donn Cave, University of Washington
+ */
+
+ /* Guess if this is AIX 4.3 */
+ #include <sys/var.h>
+ #ifdef SYSP_GET
+ #define USE_SETPRIV 1
+ #include <sys/priv.h>
+ #endif
+ #endif /* AIX */
+
#include <errno.h>

int krb5_seteuid(euid_in)
int euid_in;
{
+ int rc;
uid_t euid = (uid_t) euid_in;
+ #if defined(USE_SETPRIV)
+ priv_t pv;
+ #endif
#if defined(HAVE_SETEUID)
! rc = seteuid(euid);
! #if defined(USE_SETPRIV)
! /* setpriv actually returns int, but krb5_seteuid returns per seteuid */
! memset(&pv, 0, sizeof(pv));
! pv.pv_priv[0] |= 1 << (SET_PROC_RAC - 1);
! setpriv(PRIV_EFFECTIVE|PRIV_ADD, &pv, sizeof(pv));
! #endif
! return rc;
#else
#if defined(HAVE_SETRESUID)
return (setresuid(getuid(), euid, geteuid()));

From: Sam Hartman <hartmans@MIT.EDU>
To: donn@u.washington.edu
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-libs/908: AIX 4.3 seteuid broken
Date: 21 Dec 2000 23:31:49 -0500

I strongly recommend against accepting AIX specific privilege handling
patches. AIX 's extended privilege system was under the 3.x release
of AIX and certainly under the 4.3.3 documentation I examined not a
public interface. The intent from 3.2.5 on was to have seteuid behave
as application porters expect.

Especially since IBM has accepted the bug, I suggest that we do not do
anything special on AIX.

[end email that was filed as krb5-libs/908]