Ticket History #257: kadmin/passwd is missing USE_KDB5_LIBRARY

From tytso@MIT.EDU Wed Nov 27 11:52:16 1996
Received: from MIT.EDU (SOUTH-STATION-ANNEX.MIT.EDU [18.72.1.2]) by rt-11.MIT.EDU (8.7.5/8.7.3) with SMTP id LAA25732 for <bugs@RT-11.MIT.EDU>; Wed, 27 Nov 1996 11:52:16 -0500
Received: from DCL.MIT.EDU by MIT.EDU with SMTP
id AA06977; Wed, 27 Nov 96 11:52:12 EST
Received: by dcl.MIT.EDU (5.x/4.7) id AA08491; Wed, 27 Nov 1996 11:52:02 -0500
Message-Id: <9611271652.AA08491@dcl.MIT.EDU>
Date: Wed, 27 Nov 1996 11:52:02 -0500
From: tytso@MIT.EDU
Reply-To: tytso@MIT.EDU
To: krb5-bugs@MIT.EDU
Subject: kadmin/passwd is missing USE_KDB5_LIBRARY
X-Send-Pr-Version: 3.99

mit

System: SunOS dcl 5.4 Generic_101945-37 sun4m sparc



This is a shared library screw because libkadm5clnt requres libkdb5
(why, I'm not sure). If libkdb5 is dragged in because the
libkadm5clnt shared library requires it, on Linux all unresolved
references (in this case, the berk_db library) have to resolved at run
time or the program won't run. If libkdb5 is explicitly mentioned on
the link line, unresolved references are OK. This is arguably a bug
in the Linux's run-time linker, since Solaris apparently manages to
get this one right.

The other question is why libkadm5clnt requires libkdb5 in the first
place. It's not clear that it does, and removing libkdb5 is probably
the correct correct long-term solution (why should clients need to
have the libkdb5 shared library installed on their system?) However,
the lowest risk fix is to just add USE_KDB5_LIBRARY to the kpasswd
configure.in file.

The other question is why hasn't anyone noticed this before? I don't
know if this is a recent change to the Linux shared library loader, or
whether no one has simply ever tried running kpasswd under Linux
before. I only caught it because of the kpasswd unit tests, which I hadn't
been runing until now. (Mea culpa.)

This is, unfortunately, serious enough for us to require yet another
minor code thaw. Argh.....




From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
To: tytso@MIT.EDU, hartmans@MIT.EDU
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-admin/257: kadmin/passwd is missing USE_KDB5_LIBRARY
Date: Wed, 27 Nov 1996 12:03:03 -0500

I see what's going on. This was a recently introduced problem, checked
in by Sam to fix PR #228. I'm not sure why USE_KDB5_LIBRARY causes a
problem under netbsd --- PR#228 claims merely that casting the return
value of getuid() to an int is when calling getpwuid() is "wrong". What
this has to do with linking against libdb is beyond me.

Sam? Can we back out your change for PR#228? The severity of 228 is
marked "non-critical", so I assume this was only an aesthetic change for
NetBSD? The problem is that it's causing kpasswd to completely fail to
run under Linux.....

Failing that, there still is the question of why libkadm5clnt requires
libkdb5 (and by extension libdb) at all.

- Ted



From: "Barry Jaspan" <bjaspan@MIT.EDU>
To: krb5-bugs@MIT.EDU, tytso@MIT.EDU
Cc: Subject: Re: krb5-admin/257: kadmin/passwd is missing USE_KDB5_LIBRARY
Date: Wed, 27 Nov 1996 17:19:41 GMT

I didn't entirely follow your explanation of what exactly is going
wrong here. But I can tell you why kadm5clnt needs libkdb5: because
it uses krb5_tl_* functions, which are in that library. I *think* I
recall checking to make sure that the .o's pulled in for the tl
functions did not pull in other db code that is not needed on
client-side apps, but I cannot swear that I did so.

Potential solutions are:

o What you said, which you said was easy.

o Next easiest, duplicating the (probably small number of) tl
functions in libkdb5 that are needed in kadm5clnt directly in
libkadm5clnt so the kdb5 library is not needed.

o Pulling the tl functions out of libkdb5 entirely, since they are not
longer just a kdb5 phenomenon. This is probably the right solution,
but not for 1.0. So, leave this PR open after we finish whatever
change we're making for 1.0.

Barry


From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
To: "Barry Jaspan" <bjaspan@MIT.EDU>
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-admin/257: kadmin/passwd is missing USE_KDB5_LIBRARY
Date: Wed, 27 Nov 1996 13:36:49 -0500

The problem is that we're getting screwed by shared libraries.

Under NetBSD, pulling in libkdb5 explicitly (by including -lkdb on the
link line) causes libkdb5.a to pull in libdb.a (even though it doesn't
need to use it for any of the tl functions); however, this causes a
problem since our libdb.a is incompatible with NetBSD's libdb, which is
called by getpwuid(), thus causing netbsd to blow up in a spectacular
way. By pulling in libkdb5 implicitly, under NetBSD we win, because
it doesn't pull in libdb.a.

Under Linux, pulling in libkdb5 explicitly (by including -lkdb on the
link line) works fine, because it is able to figure out that libdb.a
isn't needed, so it doesn't worry about the missing dbm_* functions.
However, if libkdb5 is pulled in implicitly, under Linux we lose,
because the resulting binary refuses to run, complaining that the dbm_*
functions aren't defined.

There are very clearly bugs in both NetBSD and Linux's shared library
mechanisms. In any case, the end result is that if we have -lkdb5 on
the link line, kpasswd fails under NetBSD and works under Linux, and we
don't have -lkdb5 on the link line, kpasswd works under NetBSD and fails
under Linux.

The solution which Sam and I have come up with which is likely to be the
safest is to do a configure test, and include USE_KDB5_LIBRARY for any
system which doesn't contain the string "bsd". This is the equivalent
of backing out Sam's change for PR#228, except for systems which use
NetBSD.

While this should be an extremely low risk, it will mean we will need to
throw away our binaries and rebuild them.

- Ted

From: "Barry Jaspan" <bjaspan@MIT.EDU>
To: tytso@MIT.EDU
Cc: krb5-bugs@MIT.EDU, tlyu@MIT.EDU
Subject: Re: krb5-admin/257: kadmin/passwd is missing USE_KDB5_LIBRARY
Date: Wed, 27 Nov 1996 15:40:19 -0500

I've looked into this a little more and have concluded that either
I'm very confused or the problem does not actually exist.

I decided to check what in libkadm5clnt actually required libkdb5, to
see how easy it would be to duplicate the code and thus avoid the
kludge you had to make. So, I rebuild kpasswd on an SGI (with no
shared libs, obviously) without libkdb5.a expecting to see unresolved
references, and *it linked without error*. Indeed, I can find no
reference in libkadm5clnt.a to any tl-data function requiring
libkdb5.a.

So, now wondering why I ever added libkdb5.a to this link line to
begin with, I checked kpasswd's ChangeLog and found this entry:


Sure enough, kpasswd was also linking against libdb.a (via explicit
path, not -ldb, for whatever that's worth). I have no idea why Tom
made this change; perhaps it was necessary for kpasswd.local, which we
no longer build, I'm not sure. Anyway, I removed libdb from the link
line too, and again it linked without error. So then I ran cvs update
from the top of the tree and did a full rebuild, making sure that
kpasswd linked without either of libkdb5 or libdb, and ran the kpasswd
unit tests, which passed.

Therefore, Conclusion #1, I can find no evidence *with static
libraries* that libkdb5 and libdb are needed by kpasswd. If they are
still needed for shared libraries, then clearly I do not understand
shared libraries as well as I thought. But I suspect we can fix the
entire problem with kpasswd simply by removing USE_KDB5_LIBRARY from
its configure.in.

However, I note that the kadmin cli *also* links against libkdb5.a and
libdb.a. There are no automated tests for kadmin, so it seems
possible that it is also broken but no one has noticed. The kadmin
configure.in does need USE_KDB5_LIBRARY because kadmin.local links
against libkadm5srv.a, but I verified that a kadmin client linked
without libkdb5.a or libdb.a works fine. So, Conclusion #2, we should
make sure that whatever is wrong with kpasswd is not also wrong with
kadmin.

Barry

From: "Barry Jaspan" <bjaspan@MIT.EDU>
To: tytso@MIT.EDU
Cc: krb5-bugs@MIT.EDU, tlyu@MIT.EDU
Subject: Re: krb5-admin/257: kadmin/passwd is missing USE_KDB5_LIBRARY
Date: Wed, 27 Nov 1996 15:59:55 -0500

At Ted's suggestion, I also performed the following test:

% nm -o libkdb5.a | grep Proc | grep Text | perl -ne 'chop; split; print "$_[$#_]\n"' | sort > /tmp/kdb-list
% nm -o libkadm5clnt.a | grep -i proc | grep -i undefined | awk '{print $7}' | sort > /tmp/clnt-list
% comm -12 /tmp/kdb-list /tmp/clnt-list

It produced no output, indicating that no symbol that is undefined in
libkadm5clnt.a is defined in libkdb5.a.

Barry

State-Changed-From-To: open-suspended
State-Changed-By: tlyu
State-Changed-When: Sat Nov 30 17:45:47 1996
State-Changed-Why:

State-Changed-From-To: suspended-analyzed
State-Changed-By: tlyu
State-Changed-When: Sat Nov 30 17:46:11 1996
State-Changed-Why:

Lowered priority as well; we've got something that appears to work
even though it's kludgey.

State-Changed-From-To: analyzed-closed
State-Changed-By: tlyu
State-Changed-When: Wed Feb 12 21:27:41 1997
State-Changed-Why:

Not really an issue anymore; client library shouldn't really depend on
libdb2 at all, and appears to be fixed on the trunk with merged
new library build procedure.