Ticket History #372: rcache speedup via removing fsync() call

From jik@kamens.brookline.ma.us Wed Feb 12 12:21:25 1997
Received: from MIT.EDU (SOUTH-STATION-ANNEX.MIT.EDU [18.72.1.2]) by rt-11.MIT.EDU (8.7.5/8.7.3) with SMTP id MAA10197 for <bugs@RT-11.MIT.EDU>; Wed, 12 Feb 1997 12:21:25 -0500
Received: from jik.saturn.net by MIT.EDU with SMTP
id AA23990; Wed, 12 Feb 97 12:21:23 EST
Received: (from jik@localhost)
by jik.saturn.net (8.8.4/8.8.4)
id MAA23021; Wed, 12 Feb 1997 12:25:13 -0500
Message-Id: <199702121725.MAA23021@jik.saturn.net>
Date: Wed, 12 Feb 1997 12:25:13 -0500
From: "Jonathan I. Kamens" <jik@kamens.brookline.ma.us>
Reply-To: jik@cam.ov.com
To: krb5-bugs@MIT.EDU
Cc: jik@cam.ov.com
Subject: rcache speedup via removing fsync() call
X-Send-Pr-Version: 3.99

OpenVision Technologies, Inc.
N/A
The replay cache code calls fsync() to flush data to disk
after every entry it writes into the replay cache.

This has very little, if any, benefit, since once the data is
written, it will be flushed to disk soon enough unless the machine
crashes, and if the machine crashes, it could just as easily crash
before the fsync(). And besides, it'll probably take more than five
minutes to come back up, this making the replay cache irrelevant :-).

It does, however, have a significant downside -- it makes
persistent applications run slower (without the fsync, a GSS-API
context initiation takes 10ms less) and thrashes the disk
unnecessarily when an application is receiving frequent Kerberos
requests.

In short, I think the fsync() isn't necessary and should be
removed.

--- krb5-1.0/src/lib/krb5/rcache/rc_dfl.c Mon Nov 18 19:36:57 1996
+++ krb5-1.0/src/lib/krb5/rcache/rc_dfl.c Wed Feb 12 09:37:11 1997
@@ -523,13 +523,6 @@
{
return krb5_rc_dfl_expunge(context, id);
}
-#ifndef NOIOSTUFF
- else
- {
- if (krb5_rc_io_sync(context, &t->d))
- return KRB5_RC_IO;
- }
-#endif
return 0;
}


From: Tom Yu <tlyu@MIT.EDU>
To: Unassigned Problem Report <krb5-unassigned@RT-11.MIT.EDU>
Cc: krb5-bugs@MIT.EDU
Subject: Re: krb5-libs/372: rcache speedup via removing fsync() call
Date: Wed, 12 Feb 1997 17:12:46 -0500

`Tom Yu' made changes to this PR.

*** /tmp/gnatsb0032m Wed Feb 12 17:11:24 1997
--- /tmp/gnatsc0032m Wed Feb 12 17:12:34 1997
***************
*** 11,22 ****
Reply-To: jik@cam.ov.com
To: krb5-bugs@MIT.EDU
Cc: jik@cam.ov.com
! Subject: Replay cache shouldn't fsync() after every item is written
X-Send-Pr-Version: 3.99

! >Synopsis: Replay cache shouldn't fsync() after every item is written
--- 11,22 ----
Reply-To: jik@cam.ov.com
To: krb5-bugs@MIT.EDU
Cc: jik@cam.ov.com
! Subject: rcache speedup via removing fsync() call
X-Send-Pr-Version: 3.99

! >Synopsis: rcache speedup via removing fsync() call

From: "Jonathan I. Kamens" <jik@cam.ov.com>
To: krb5-bugs@MIT.EDU
Cc: Subject: Re: krb5-libs/372: replay cache code shouldn't fsync after every entry added to the cache
Date: Mon, 11 Aug 1997 15:58:49 -0400

Has this patch been merged back into the main branch of the MIT source
tree? Will it be included in the next MIT release?

Thanks,

jik

From jik@kamens.brookline.ma.us Tue Nov 23 16:51:56 1999
Received: from MIT.EDU (PACIFIC-CARRIER-ANNEX.MIT.EDU [18.69.0.28]) by rt-11.MIT.EDU (8.7.5/8.7.3) with SMTP id QAA02254 for <bugs@RT-11.MIT.EDU>; Tue, 23 Nov 1999 16:51:55 -0500
Received: from nautilus.shore.net by MIT.EDU with SMTP
id AA05127; Tue, 23 Nov 99 16:52:28 EST
Received: from jik.shore.net [206.243.167.15]
by nautilus.shore.net with esmtp (Exim)
for krb5-bugs@mit.edu
id 11qNqh-0002Z9-00; Tue, 23 Nov 1999 16:51:43 -0500
Received: from jik2.kamens.brookline.ma.us (IDENT:root@jik2.kamens.brookline.ma.us [192.168.0.3])
by jik.shore.net (8.9.0/8.9.0) with ESMTP id QAA21171
for <krb5-bugs@mit.edu>; Tue, 23 Nov 1999 16:51:06 -0500
Received: (from jik@localhost)
by jik2.kamens.brookline.ma.us (8.9.3/8.9.3) id QAA21023;
Tue, 23 Nov 1999 16:51:58 -0500
Message-Id: <199911232151.QAA21023@jik2.kamens.brookline.ma.us>
Date: Tue, 23 Nov 1999 16:51:58 -0500
From: Jonathan Kamens <jik@kamens.brookline.ma.us>
Reply-To: jik@kamens.brookline.ma.us
To: krb5-bugs@MIT.EDU
Subject: Replay cache code shouldn't call fsync() after every entry is written
X-Send-Pr-Version: 3.99



System: Linux jik2 2.2.12-20 #14 Thu Nov 18 20:57:02 EST 1999 i686 unknown
Architecture: i686

The replay cache code shouldn't call fsync() after every entry
is written into the replay cache. There's no point to it, and
it significantly hurts performance and thrashes the disk
unnecessarily when a server is receiving a lot of requests.

Index: ./src/lib/krb5/rcache/rc_dfl.c
--- rc_dfl.c 1999/11/23 20:50:58 1.1.1.1
+++ rc_dfl.c 1999/11/23 21:36:37 1.3
@@ -532,13 +532,6 @@
{
return krb5_rc_dfl_expunge(context, id);
}
-#ifndef NOIOSTUFF
- else
- {
- if (krb5_rc_io_sync(context, &t->d))
- return KRB5_RC_IO;
- }
-#endif
return 0;
}


From: tytso@MIT.EDU
To: jik@kamens.brookline.ma.us
Cc: krb5-bugs@MIT.EDU, krb5-unassigned@RT-11.MIT.EDU,
gnats-admin@RT-11.MIT.EDU, krb5-prs@RT-11.MIT.EDU
Subject: Re: krb5-libs/787: Replay cache code shouldn't call fsync() after every entry is written
Date: Mon, 29 Nov 1999 14:25:11 -0500

Date: Tue, 23 Nov 1999 16:51:58 -0500
From: Jonathan Kamens <jik@kamens.brookline.ma.us>

The replay cache code shouldn't call fsync() after every entry
is written into the replay cache. There's no point to it, and
it significantly hurts performance and thrashes the disk
unnecessarily when a server is receiving a lot of requests.

There's no question that having the replay cache call fsync() causes
disk activity. The original reasoning behind this was that most of the
time servers aren't making huge numbers of authentications per second,
and the furthermore, the server should be protected against replays in
case of server crash.

Of course, it's all a tradeoff; on systems with journalled filesystems
(so the fsck time can be avoided), a system can reboot faster than the 5
minute clock skew allowance. Then again, these circumstances don't
happen that often; so perhaps for some applications, not calling fsync()
is the right thing. It's not so obvious to me that fscync() should
*always* be avoided, though.

- Ted