Skip Menu |
 

From: Sam Hartman <hartmans@debian.org>
To: krb5-bugs@MIT.EDU
Subject: gssapi mechanism glue dlcloses objects potentially after they are already unloaded
Date: Wed, 16 May 2012 20:41:37 -0400
CC: hartmans@MIT.EDU
Download (untitled) / with headers
text/plain 1.6KiB


moonshot has consistently gotten a linker error on Linux at process exit
time. I traced it with a debugger today and figured out what seems to be
going on.

exit calls the library finalization functions for loaded libraries.
That calls gssint_mechglue_fini. If PROGRAM_EXITING() returns false
(which it always does) that calls freeMechList() which indirectly calls
dlclose on loaded objects.

Unfortunately, moonshot depends circularly on libgssapi_krb5. The linker
chooses to break this dependency by finalizing moonshot first.
Then when we call dlclose, we get:

Inconsistency detected by ld.so: dl-close.c: 743: _dl_close: Assertion
`map->l_i nit_called' failed!


This is arguably a linker bug.

A few observations:

1) It would be nice to have a better definition of PROGRAM_EXITING for
glibc platforms.

2) There are reasonably good reasons for GSS mechanisms to call into
libgssapi_krb5 even though they are loaded by it.

Finally, I suspect that gssint_mechglue_fini may tend to be useless if a
mechanism that calls into the mechglue is loaded. Consider what
happens. The cases where fini matters is where the gss library would
like to be unloaded. So, an application dlopens a plugin that loads
gssapi_krb5. That eventually opens moonshot, which creates a second
reference to libgssapi_krb5.

The application dlcloses the plugin, removing one reference to
libgssapi_krb5. The linker really needs to garbage collect or otherwise
walk the graph in order to determine that it can get rid of the
moonshot-gssapi subgraph. So on Linux, it looks a lot like this code
crashes at process exit and is likely to be useless otherwise.
We decided to start using RTLD_NODELETE when loading plugin modules,
mostly for OpenSSL-related reasons. I think that should fix this
problem; I would be interested in test results.

Of course, this change also makes it even more pointless to dlclose() GSS
modules from the libgssapi_krb5 finalizer. But we need to at least free
the struct plugin_file_handle to avoid a memory leak on repeated loading
and unloading of libgssapi_krb5, and there is also Windows to consider.
The RTLD_NODELETE change (ticket 7947) does not appear to solve this
problem; in fact, it's possible that it makes it manifest under
different circumstances.

However, the upstream glibc bug has now been fixed. References:

https://sourceware.org/git/?
p=glibc.git;a=commit;h=57707b7fcc38855869321f8c7827bfe21d729f37
https://www.sourceware.org/ml/libc-alpha/2016-12/msg00859.html
https://bugzilla.redhat.com/show_bug.cgi?id=1398370
Although this was fixed in glibc 2.25, we now have a need to work around it for systems with older glibc versions.  (Otherwise "make check" will break on reasonably current Linux distributions due to the negoex tests.)  I am reopening this and clearing the nochange tag.
 
From: ghudson@mit.edu
Subject: git commit

Work around glibc bug 11941 (dlclose assertion)

When building against glibc 2.24 or earlier, suppress calls to
dlclose() to prevent the assertion failure "_dl_close: Assertion
`map->l_init_called' failed" at process exit. We need this workaround
to enable automated tests that load GSSAPI modules.

https://github.com/krb5/krb5/commit/bf5953c549a6d279977df69ffe89b2ba51460eaf
Author: Greg Hudson <ghudson@mit.edu>
Commit: bf5953c549a6d279977df69ffe89b2ba51460eaf
Branch: master
src/util/support/plugins.c | 15 +++++++++++++++
1 files changed, 15 insertions(+), 0 deletions(-)