RT RT/krbdev.mit.edu: Ticket #372 rcache speedup via removing fsync() call Signed in as guest.
[Logout]

[Home] [Search] [Configuration]

[Display] [History] [Basics] [Dates] [People] [Links] [Jumbo]

 
 

 The Basics  
Id
372
Status
open
Worked
0 min
Priority
25/
Queue
krb5
 

 Keyword Selections  
Component
  • krb5-libs
Tags
  • enhancement
Version_reported
Version_Fixed
Target_Version
 

 Relationships  
Depends on:
Depended on by:
Parents:
  • 1784: (raeburn) replay cache performance sucks [resolved]
Children:

Refers to:
Referred to by:
 
 Dates  
Created: Wed Feb 12 12:22:01 1997
Starts: Not set
Started: Wed Aug 27 00:20:41 2003
Last Contact: Not set
Due: Not set
Updated: Tue Jul 1 15:51:42 2008 by tlyu
 

 People  
Owner
 Nobody
Requestors
 jik@kamens.brookline.ma.us
Cc
 
AdminCc
 
 

 More about jik@kamens.brookline.ma.us  
Comments about this user:
No comment entered about this user
This user's 25 highest priority tickets:
 

History   Display mode: [Brief headers] [Full headers]
      Mon Aug 19 14:20:07 2002  RT_System - Default: Import/ changed from to    
     
From jik@kamens.brookline.ma.us  Wed Feb 12 12:21:25 1997
Received: from MIT.EDU (SOUTH-STATION-ANNEX.MIT.EDU [18.72.1.2]) by rt-11.MIT.EDU
(8.7.5/8.7.3) with SMTP id MAA10197 for <bugs@RT-11.MIT.EDU>; Wed, 12 Feb 1997
12:21:25 -0500
Received: from jik.saturn.net by MIT.EDU with SMTP
	id AA23990; Wed, 12 Feb 97 12:21:23 EST
Received: (from jik@localhost)
          by jik.saturn.net (8.8.4/8.8.4)
	  id MAA23021; Wed, 12 Feb 1997 12:25:13 -0500
Message-Id: <199702121725.MAA23021@jik.saturn.net>
Date: Wed, 12 Feb 1997 12:25:13 -0500
From: "Jonathan I. Kamens" <jik@kamens.brookline.ma.us>
Reply-To: jik@cam.ov.com
To: krb5-bugs@MIT.EDU
Cc: jik@cam.ov.com
Subject: rcache speedup via removing fsync() call
X-Send-Pr-Version: 3.99

>Number:         372
>Category:       krb5-libs
>Synopsis:       rcache speedup via removing fsync() call
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    krb5-unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   unknown
>Arrival-Date:   Wed Feb 12 12:22:01 EST 1997
>Last-Modified:  Mon Aug 11 15:59:00 EDT 1997
>Originator:     Jonathan I. Kamens
>Organization:
OpenVision Technologies, Inc.
>Release:        1.0
>Environment:
N/A
>Description:
	The replay cache code calls fsync() to flush data to disk
after every entry it writes into the replay cache.

	This has very little, if any, benefit, since once the data is
written, it will be flushed to disk soon enough unless the machine
crashes, and if the machine crashes, it could just as easily crash
before the fsync().  And besides, it'll probably take more than five
minutes to come back up, this making the replay cache irrelevant :-).

	It does, however, have a significant downside -- it makes
persistent applications run slower (without the fsync, a GSS-API
context initiation takes 10ms less) and thrashes the disk
unnecessarily when an application is receiving frequent Kerberos
requests.

	In short, I think the fsync() isn't necessary and should be
removed.
>How-To-Repeat:

>Fix:
--- krb5-1.0/src/lib/krb5/rcache/rc_dfl.c	Mon Nov 18 19:36:57 1996
+++ krb5-1.0/src/lib/krb5/rcache/rc_dfl.c	Wed Feb 12 09:37:11 1997
@@ -523,13 +523,6 @@
     {
    return krb5_rc_dfl_expunge(context, id);
     }
-#ifndef NOIOSTUFF
-    else
-    {
-	if (krb5_rc_io_sync(context, &t->d))
-	    return KRB5_RC_IO;
-    }
-#endif
  return 0;
 }

>Audit-Trail:

From: Tom Yu <tlyu@MIT.EDU>
To: Unassigned Problem Report <krb5-unassigned@RT-11.MIT.EDU>
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-libs/372: rcache speedup via removing fsync() call
Date: Wed, 12 Feb 1997 17:12:46 -0500

 	`Tom Yu' made changes to this PR.

 *** /tmp/gnatsb0032m	Wed Feb 12 17:11:24 1997
 --- /tmp/gnatsc0032m	Wed Feb 12 17:12:34 1997
 ***************
 *** 11,22 ****
   Reply-To: jik@cam.ov.com
   To: krb5-bugs@MIT.EDU
   Cc: jik@cam.ov.com
 ! Subject: Replay cache shouldn't fsync() after every item is written
   X-Send-Pr-Version: 3.99

   >Number:         372
   >Category:       krb5-libs
 ! >Synopsis:       Replay cache shouldn't fsync() after every item is written
   >Confidential:   no
   >Severity:       non-critical
   >Priority:       low
 --- 11,22 ----
   Reply-To: jik@cam.ov.com
   To: krb5-bugs@MIT.EDU
   Cc: jik@cam.ov.com
 ! Subject: rcache speedup via removing fsync() call
   X-Send-Pr-Version: 3.99

   >Number:         372
   >Category:       krb5-libs
 ! >Synopsis:      rcache speedup via removing fsync() call
   >Confidential:   no
   >Severity:       non-critical
   >Priority:       low

From: "Jonathan I. Kamens" <jik@cam.ov.com>
To: krb5-bugs@MIT.EDU
Cc:  Subject: Re: krb5-libs/372: replay cache code shouldn't fsync after every entry
added to the cache
Date: Mon, 11 Aug 1997 15:58:49 -0400

 Has this patch been merged back into the main branch of the MIT source
 tree?  Will it be included in the next MIT release?

 Thanks,

   jik
>Unformatted:


Download (untitled) 3.8k
      Mon Aug 19 14:20:07 2002  RT_System - Component krb5-libs added    
      Mon Aug 19 14:20:50 2002  RT_System - Ticket 787: Default: Import/ changed from to    
     
From jik@kamens.brookline.ma.us  Tue Nov 23 16:51:56 1999
Received: from MIT.EDU (PACIFIC-CARRIER-ANNEX.MIT.EDU [18.69.0.28]) by rt-11.MIT.EDU
(8.7.5/8.7.3) with SMTP id QAA02254 for <bugs@RT-11.MIT.EDU>; Tue, 23 Nov 1999
16:51:55 -0500
Received: from nautilus.shore.net by MIT.EDU with SMTP
	id AA05127; Tue, 23 Nov 99 16:52:28 EST
Received: from jik.shore.net [206.243.167.15]
	by nautilus.shore.net with esmtp (Exim)
	for krb5-bugs@mit.edu
	id 11qNqh-0002Z9-00; Tue, 23 Nov 1999 16:51:43 -0500
Received: from jik2.kamens.brookline.ma.us (IDENT:root@jik2.kamens.brookline.ma.us
[192.168.0.3])
	by jik.shore.net (8.9.0/8.9.0) with ESMTP id QAA21171
	for <krb5-bugs@mit.edu>; Tue, 23 Nov 1999 16:51:06 -0500
Received: (from jik@localhost)
	by jik2.kamens.brookline.ma.us (8.9.3/8.9.3) id QAA21023;
	Tue, 23 Nov 1999 16:51:58 -0500
Message-Id: <199911232151.QAA21023@jik2.kamens.brookline.ma.us>
Date: Tue, 23 Nov 1999 16:51:58 -0500
From: Jonathan Kamens <jik@kamens.brookline.ma.us>
Reply-To: jik@kamens.brookline.ma.us
To: krb5-bugs@MIT.EDU
Subject: Replay cache code shouldn't call fsync() after every entry is written
X-Send-Pr-Version: 3.99

>Number:         787
>Category:       krb5-libs
>Synopsis:       Replay cache code shouldn't call fsync() after every entry is
written
>Confidential:   yes
>Severity:       non-critical
>Priority:       medium
>Responsible:    krb5-unassigned
>State:          open
>Class:          change-request
>Submitter-Id:   unknown
>Arrival-Date:   Tue Nov 23 16:52:00 EST 1999
>Last-Modified:  Mon Nov 29 14:27:01 EST 1999
>Originator:     Jonathan Kamens
>Organization:

>Release:        krb5-1.1
>Environment:

System: Linux jik2 2.2.12-20 #14 Thu Nov 18 20:57:02 EST 1999 i686 unknown
Architecture: i686

>Description:
	The replay cache code shouldn't call fsync() after every entry
	is written into the replay cache.  There's no point to it, and
	it significantly hurts performance and thrashes the disk
	unnecessarily when a server is receiving a lot of requests.
>How-To-Repeat:

>Fix:
Index: ./src/lib/krb5/rcache/rc_dfl.c
--- rc_dfl.c	1999/11/23 20:50:58	1.1.1.1
+++ rc_dfl.c	1999/11/23 21:36:37	1.3
@@ -532,13 +532,6 @@
     {
    return krb5_rc_dfl_expunge(context, id);
     }
-#ifndef NOIOSTUFF
-    else
-    {
-	if (krb5_rc_io_sync(context, &t->d))
-	    return KRB5_RC_IO;
-    }
-#endif
  return 0;
 }

>Audit-Trail:

From: tytso@MIT.EDU
To: jik@kamens.brookline.ma.us
Cc: krb5-bugs@MIT.EDU, krb5-unassigned@RT-11.MIT.EDU,
        gnats-admin@RT-11.MIT.EDU, krb5-prs@RT-11.MIT.EDU
Subject: Re: krb5-libs/787: Replay cache code shouldn't call fsync() after every
entry is written
Date: Mon, 29 Nov 1999 14:25:11 -0500

    Date: Tue, 23 Nov 1999 16:51:58 -0500
    From: Jonathan Kamens <jik@kamens.brookline.ma.us>

 	   The replay cache code shouldn't call fsync() after every entry
 	   is written into the replay cache.  There's no point to it, and
 	   it significantly hurts performance and thrashes the disk
 	   unnecessarily when a server is receiving a lot of requests.

 There's no question that having the replay cache call fsync() causes
 disk activity.  The original reasoning behind this was that most of the
 time servers aren't making huge numbers of authentications per second,
 and the furthermore, the server should be protected against replays in
 case of server crash.

 Of course, it's all a tradeoff; on systems with journalled filesystems
 (so the fsck time can be avoided), a system can reboot faster than the 5
 minute clock skew allowance.  Then again, these circumstances don't
 happen that often; so perhaps for some applications, not calling fsync()
 is the right thing.  It's not so obvious to me that fscync() should
 *always* be avoided, though.

 						- Ted
>Unformatted:


Download (untitled) 3.6k
      Mon Aug 19 14:20:50 2002  RT_System - Ticket 787: Tags enhancement added    
      Mon Aug 19 14:20:50 2002  RT_System - Ticket 787: Component krb5-libs added    
      Mon Aug 19 14:20:50 2002  RT_System - Ticket 787: Version_reported 1.1 added    
      Wed Aug 27 00:20:41 2003  raeburn - Status changed from new to open    
      Wed Aug 27 00:20:42 2003  raeburn - Comments added    
     
Subject: rcache speedup via removing fsync() call

> 	This has very little, if any, benefit, since once the data is
> written, it will be flushed to disk soon enough unless the machine
> crashes, and if the machine crashes, it could just as easily crash
> before the fsync().  And besides, it'll probably take more than five
> minutes to come back up, this making the replay cache irrelevant :-).

Unfortunately(?), some systems can come back up much quicker than that.
But maybe we can figure out some way to deal...  Tom and I kicked around
a few ideas, but didn't get to anything beyond either doing the fsync,
skipping the replay cache altogether because the protocol makes it
unnecessary, or suffering with the replay vulnerability.

I did a little testing tonight, using a program that just adds to the
replay cache as quickly as it can for about five minutes, and when I
disabled the fsync call, the program ran faster by about a factor of
four, and it didn't keep the Linux kjournald process busy the whole
time.  But even with the fsync call, I was able to add around 1200
replay cache entries per second on average.  (Starting with an empty
cache, not running long enough to trigger an expunge, etc...not a
perfect test by any means.)


Download (untitled) 1.1k
      Fri Jan  9 16:56:24 2004  hartmans - Ticket 787: Subject changed from Replay cache code shouldn't call fsync() after every entry is written to preauth    
      Thu Dec  9 19:36:52 2004  tlyu - Ticket 787: Subject changed from Replay cache code shouldn't call fsync() after every entry is written to Replay cache code shouldn't call fsync() after every entry is written    
      Thu Dec  9 19:36:53 2004  tlyu - Ticket 787: Status changed from new to open    
      Thu Dec  9 19:36:53 2004  tlyu - Ticket 787: Comments added    
     
We probably should eventually have an option to select whether to
fsync() the rcache on updates.


Download (untitled) 96b
      Wed Feb  7 16:17:29 2007  tlyu - Subject changed from rcache speedup via removing fsync() call to rcache speedup via removing fsync() call    
      Wed Feb  7 16:17:29 2007  tlyu - Tags enhancement added    
      Tue Jul  1 15:51:41 2008  tlyu - Ticket fsck.com-rt://mit.edu/krbdev.mit.edu/ticket/1784 no longer RefersTo ticket 372.    
      Tue Jul  1 15:51:42 2008  tlyu - Ticket 372 MemberOf ticket 1784.    
      Tue Dec 31 14:27:52 2013  tlyu - Ticket 787: Ticket 787 MergedInto ticket 372.