From: Luke Howard <lukeh@padl.com>
To: krb5-bugs@mit.edu
Subject: libkrb5 treats all KDC errors as terminal
Date: Wed, 11 Jan 2006 07:56:21 +1100


If a KDC returns KDC_ERR_SVC_UNAVAILABLE, it appears that that the
client library will not failover to the next KDC. All error codes,
except for KDC_ERR_RESPONSE_TOO_BIG, are treated as terminal.

This won't be completely trivial to fix but I will look into it as
it causing a real problem at a customer.

-- Luke

--

To: rt@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
From: Tom Yu <tlyu@MIT.EDU>
Date: Tue, 10 Jan 2006 16:37:46 -0500
RT-Send-Cc: 

>>>>> "lukeh" == lukeh@padl com via RT <rt-comment@krbdev.mit.edu> writes:

lukeh> If a KDC returns KDC_ERR_SVC_UNAVAILABLE, it appears that that the
lukeh> client library will not failover to the next KDC. All error codes,
lukeh> except for KDC_ERR_RESPONSE_TOO_BIG, are treated as terminal.

Could you be more specific about whether this is for AS_REQ or
TGS_REQ, and about which functions are affected?

Also, note that our code doesn't actually define
KDC_ERR_SVC_UNAVAILABLE.  Do you know what its semantics are?  Are
there other KDC error codes you feel should be "soft" errors?

---Tom

From: Ken Raeburn <raeburn@MIT.EDU>
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal 
Date: Tue, 10 Jan 2006 18:22:12 -0500
To: rt@krbdev.mit.edu
RT-Send-Cc: 

On Jan 10, 2006, at 16:06, lukeh@padl.com via RT wrote:
> If a KDC returns KDC_ERR_SVC_UNAVAILABLE, it appears that that the
> client library will not failover to the next KDC. All error codes,
> except for KDC_ERR_RESPONSE_TOO_BIG, are treated as terminal.


Looks like this error code was quietly added from RFC 1510 -> RFC
4120 (somewhere between draft 03, 11/98, and draft 04, 6/99), without
any definition besides the not-very-helpful text "A service is not
available."  There is no indication in the main text when this error
code might be returned.  If it said "the KDC service is not
available", that might've been clearer, but "*a* service"??  What,
should we return this error if the local SMTP handler has been shut
down?

Also, the MIT code has no concept of "failing over to the next KDC".
If it gets a response back from any KDC, the loop querying the KDCs
terminates.  RESPONSE_TOO_BIG is handled by starting the whole
process up again but looking only for TCP servers.

Handling this properly -- assuming failing over to the next KDC is in
fact the right response to this error code -- will probably involve
rewriting the query loop to add a callback to examine the returned
data and decide whether or how to continue the current query loop,
which servers to drop, etc.  This would also be a better way of
handling RESPONSE_TOO_BIG, as well.

Ken

To: rt@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
From: Sam Hartman <hartmans@mit.edu>
Date: Wed, 11 Jan 2006 15:39:43 -0500
RT-Send-Cc: 

FYI, we've indicated to Luke in private mail that we would accept a
patch for this.

From: Luke Howard <lukeh@padl.com>
To: rt-comment@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Thu, 12 Jan 2006 08:27:32 +1100
RT-Send-Cc: 


I do have a patch, it's not terribly elegant nor does it handle
the TCP fallback case. Essentially the callback is invoked from
krb5int_sendto() and will ignore the response if it indicates
KRB5KDC_ERR_SVC_UNAVAILABLE.

Of course, it's going to be a little hard to test because the
KDC (Active Directory) returns the error only in very rare, and
difficult to reproduce, circumstances.

Interestingly, see bug #2738.

-- Luke

>From: "Sam Hartman via RT" <rt-comment@krbdev.mit.edu>
>Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
>To: lukeh@padl.com
>Date: Wed, 11 Jan 2006 15:40:06 -0500 (EST)
>
>FYI, we've indicated to Luke in private mail that we would accept a
>patch for this.
>

--

From: Luke Howard <lukeh@padl.com>
To: rt-comment@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Tue, 17 Jan 2006 03:53:32 +1100
RT-Send-Cc: 


Here's the patch we tried:

http://www.padl.com/~lukeh/krb5-1.3.5-svc-unavailable.patch

Not elegant, but it appears to work (from inserting error in code path,
unfortunately we can't make the MS KDC force this error).

-- Luke

>From: "Sam Hartman via RT" <rt-comment@krbdev.mit.edu>
>Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
>To: lukeh@padl.com
>Date: Wed, 11 Jan 2006 15:40:06 -0500 (EST)
>
>FYI, we've indicated to Luke in private mail that we would accept a
>patch for this.
>

--

From: Luke Howard <lukeh@padl.com>
To: rt-comment@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Fri, 17 Mar 2006 16:37:18 +1100
RT-Send-Cc: 


>Looks like this error code was quietly added from RFC 1510 -> RFC
>4120 (somewhere between draft 03, 11/98, and draft 04, 6/99), without
>any definition besides the not-very-helpful text "A service is not
>available."  There is no indication in the main text when this error
>code might be returned.  If it said "the KDC service is not
>available", that might've been clearer, but "*a* service"??  What,
>should we return this error if the local SMTP handler has been shut
>down?

Ah, seems like it came from:

http://diswww.mit.edu:8008/menelaus.mit.edu/kprot/359

-- Luke

--

From: Luke Howard <lukeh@padl.com>
To: rt-comment@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Tue, 12 Dec 2006 12:03:30 +1100
RT-Send-Cc: 


Our customer has requested whether this patch (or something like
it) will be integrated. Did anyone ever have a change to look at
the patch I submitted?

-- Luke

>From: "Sam Hartman via RT" <rt-comment@krbdev.mit.edu>
>Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
>To: lukeh@padl.com
>Date: Wed, 11 Jan 2006 15:40:06 -0500 (EST)
>
>FYI, we've indicated to Luke in private mail that we would accept a
>patch for this.
>

--
www.padl.com | www.lukehoward.com

From: Ken Raeburn <raeburn@MIT.EDU>
Subject: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Fri, 16 Feb 2007 19:34:10 -0500
To: rt@krbdev.mit.edu
RT-Send-Cc: 

Overall, I think this is the right approach to use.  I'd suggest a
few minor(?) changes, and have some notes:

* Make the error message a little more meaningful to the user.
Frankly, I think it's kind of silly to be telling the end user about
specific internal errors in the KDC, but here we are...

* The sendto and sendto_kdc interfaces are private; we can just
change them, and the callers.  It probably means a bigger patch, but
less proliferation of internal interfaces.  (And for simplicity,
perhaps a null callback function pointer should mean to return on
receipt of any complete message.)

* After working with a few callback interfaces, I think it's a good
idea to avoid mingling "abandon this connection" flags and OS errors
or errors reported by the other party in a callback interface.  The
controlling loop is simpler if the callback can only tell it "keep
going" vs "abandon this connection" vs "we're done", and any error
indications needed but irrelevant to the controlling loop itself can
be stuffed in the callback data structure, to be checked by the
caller when the controlling loop returns.  This is especially true if
you can get errors that you might want to report depending on the
results of other calls, so your control loop would potentially have
to track multiple error conditions.

* Rename the typedef krb5_select_kdc_fct so it doesn't refer to KDCs
specifically in the sendto interface, which isn't specific to KDC
exchanges.  Maybe something like sendto_response_handler, and maybe
make the function pointer plus data pointer into a struct like was
done with struct sendto_callback_info (which perhaps now should be
renamed since it's not the only callback) for constructing per-server
messages.

* Someday, we could make sendto_kdc return the error structure we've
already parsed (to see if it was SVC_UNAVAILABLE).  Currently,
sendto_kdc just passes back a krb5_data.  Just noting this for the
future...

* We could also check for RESPONSE_TOO_BIG, and shut down only the
one UDP socket.  Our current behavior is to quit out of the whole
sendto loop when we get the response, and shut down all the UDP and
TCP sockets; then we analyze the returned message, see that it's
RESPONSE_TOO_BIG, look up the TCP KDC service again, and start up the
TCP connections again.  That's pretty wasteful.  Keeping the other
UDP sockets open (and, in fact, possibly opening new UDP sockets)
until we get back RESPONSE_TOO_BIG on them is also wasteful, but
perhaps not quite as bad.

Ken

From: Ken Raeburn <raeburn@MIT.EDU>
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal
Date: Mon, 21 May 2007 12:04:47 -0400
To: rt-comment@krbdev.mit.edu
RT-Send-Cc: 

Ignore this comment, I'm testing a change to the mailer configuration...

Ken,

What's the status on this -- are you waiting for me to provide a
candidate implementation?

I can't remember where we left it.

-- Luke

From: Ken Raeburn <raeburn@MIT.EDU>
Subject: Re: [krbdev.mit.edu #3334] libkrb5 treats all KDC errors as terminal 
Date: Mon, 11 Jun 2007 19:51:53 -0400
To: rt@krbdev.mit.edu
RT-Send-Cc: 

> Ken,
>
> What's the status on this -- are you waiting for me to provide a
> candidate implementation?
>
> I can't remember where we left it.
>
> -- Luke


Ah, I'd forgotten about this.  I think I was waiting to see if you
were going to revise your patch to address my suggestions, or not....

Ken

New untested patch worked up with Luke, just getting it recorded someplace...

From: raeburn@mit.edu
Subject: SVN Commit

Debugged version of patch worked up with Luke.

Adds a callback to krb5int_sendto to examine the response and indicate
whether to quit the loop or not.  For sendto_kdc, keep going if the
returned error is "service unavailable".  Updated all other callers to
pass a null function pointer, which means to always break out of the
loop on any response (the old behavior).


Commit By: raeburn



Revision: 19738
Changed Files:
_U  trunk/
U   trunk/src/include/k5-int.h
U   trunk/src/lib/krb4/send_to_kdc.c
U   trunk/src/lib/krb5/error_tables/krb5_err.et
U   trunk/src/lib/krb5/os/changepw.c
U   trunk/src/lib/krb5/os/send524.c
U   trunk/src/lib/krb5/os/sendto_kdc.c

The change to the signature of an accessor function probably requires
bumping the version number on the accessor.

From: tlyu@mit.edu
Subject: SVN Commit

bump accessor version number

Commit By: tlyu



Revision: 19911
Changed Files:
_U  trunk/
U   trunk/src/include/k5-int.h

To: rt@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] SVN Commit
From: Tom Yu <tlyu@MIT.EDU>
Date: Mon, 03 Sep 2007 23:59:32 -0400
RT-Send-Cc: 

This doesn't merge cleanly to the 1.6 branch.  There appears to be a
missing prereq patch to src/lib/krb5/os/sendto_kdc.c.

Date: Wed, 05 Sep 2007 12:17:12 -0600
From: Luke Howard <lukeh@padl.com>
To: rt-comment@krbdev.mit.edu
Subject: Re: [krbdev.mit.edu #3334] SVN Commit
RT-Send-Cc: 

Ken has the current patch, so you'd need to ask him.

Tom Yu via RT wrote:
> This doesn't merge cleanly to the 1.6 branch.  There appears to be a
> missing prereq patch to src/lib/krb5/os/sendto_kdc.c.
>
>
>


--
www.padl.com | www.lukehoward.com

From: Ken Raeburn <raeburn@MIT.EDU>
Subject: Re: [krbdev.mit.edu #3334] SVN Commit
Date: Thu, 6 Sep 2007 18:09:49 -0400
To: rt-comment@krbdev.mit.edu
RT-Send-Cc: 

> This doesn't merge cleanly to the 1.6 branch.  There appears to be a
> missing prereq patch to src/lib/krb5/os/sendto_kdc.c.

Looks like r19456 is needed too.

Ken

From: tlyu@mit.edu
Subject: SVN Commit

pull up r19456 from trunk (prereq for r19738)

 r19456@cathode-dark-space:  raeburn | 2007-04-13 01:44:38 -0400
 Produce a more informative error message for KDC_UNREACH with KDC not responding.


Commit By: tlyu



Revision: 19933
Changed Files:
_U  branches/krb5-1-6/
U   branches/krb5-1-6/src/lib/krb5/os/sendto_kdc.c

From: tlyu@mit.edu
Subject: SVN Commit

pull up r19738 from trunk

 r19738@cathode-dark-space:  raeburn | 2007-07-27 00:39:21 -0400
 ticket: 3334

 Debugged version of patch worked up with Luke.

 Adds a callback to krb5int_sendto to examine the response and indicate
 whether to quit the loop or not.  For sendto_kdc, keep going if the
 returned error is "service unavailable".  Updated all other callers to
 pass a null function pointer, which means to always break out of the
 loop on any response (the old behavior).




Commit By: tlyu



Revision: 19934
Changed Files:
_U  branches/krb5-1-6/
U   branches/krb5-1-6/src/include/k5-int.h
U   branches/krb5-1-6/src/lib/krb4/send_to_kdc.c
U   branches/krb5-1-6/src/lib/krb5/error_tables/krb5_err.et
U   branches/krb5-1-6/src/lib/krb5/os/changepw.c
U   branches/krb5-1-6/src/lib/krb5/os/send524.c
U   branches/krb5-1-6/src/lib/krb5/os/sendto_kdc.c

From: tlyu@mit.edu
Subject: SVN Commit

pull up r19911 from trunk

 r19911@cathode-dark-space:  tlyu | 2007-09-03 23:18:05 -0400
 ticket: 3334

 bump accessor version number



Commit By: tlyu



Revision: 19948
Changed Files:
_U  branches/krb5-1-6/
U   branches/krb5-1-6/src/include/k5-int.h