ntfs tracing

NFS Tracing By Passive Network Monitoring Matt Blaze Department of Computer Science Princeton University mab@cs.princeton.edu ABSTRACT Traces of filesystem activity have proven to be useful for a wide variety of purposes, rang ing from quantitative analysis of system behavior to trace-driven simulation of filesystem algo rithms. Such traces can be difficult to obtain, however, usually entailing modification of the filesystems to be monitored and runtime overhead for the period of the trace. Largely because of these difficulties, a surprisingly small number of filesystem traces have been conducted, and few sample workloads are available to filesystem researchers. This paper describes a portable toolkit for deriving approximate traces of NFS [1] activity by non-intrusively monitoring the Ethernet traffic to and from the file server. The toolkit uses a promiscuous Ethernet listener interface (such as the Packetfilter[2]) to read and reconstruct NFS-related RPC packets intended for the server. It produces traces of the NFS activity as well as a plausible set of corresponding client system calls. The tool is currently in use at Princeton and other sites, and is available via anonymous ftp. 1. Motivation Traces of real workloads form an important part of virtually all analysis of computer system behavior, whether it is program hot spots, memory access patterns, or filesystem activity that is being studied. In the case of filesystem activity, obtaining useful traces is particularly challenging. Filesystem behavior can span long time periods, often making it necessary to collect huge traces over weeks or even months. Modification of the filesystem to collect trace data is often difficult, and may result in unacceptable runtime overhead. Distributed filesystems exa cerbate these difficulties, especially when the network is composed of a large number of heterogeneous machines. As a result of these difficulties, only a relatively small number of traces of Unix filesystem workloads have been conducted, primarily in computing research environments. [3], [4] and [5] are examples of such traces. Since distributed filesystems work by transmitting their activity over a network, it would seem reasonable to obtain traces of such systems by placing a "tap" on the network and collecting trace data based on the network traffic. Ethernet[6] based networks lend themselves to this approach particularly well, since traffic is broadcast to all machines connected to a given subnetwork. A number of general-purpose network monitoring tools are avail able that "promiscuously" listen to the Ethernet to which they are connected; Sun's etherfind[7] is an example of such a tool. While these tools are useful for observing (and collecting statistics on) specific types of packets, the information they provide is at too low a level to be useful for building filesystem traces. Filesystem operations may span several packets, and may be meaningful only in the context of other, previous operations. Some work has been done on characterizing the impact of NFS traffic on network load. In [8], for example, the results of a study are reported in which Ethernet traffic was monitored and statistics gathered on NFS activity. While useful for understanding traffic patterns and developing a queueing model of NFS loads, these previous stu dies do not use the network traffic to analyze the file access traffic patterns of the system, focusing instead on developing a statistical model of the individual packet sources, destinations, and types. This paper describes a toolkit for collecting traces of NFS file access activity by monitoring Ethernet traffic. A "spy" machine with a promiscuous Ethernet interface is connected to the same network as the file server. Each NFS-related packet is analyzed and a trace is produced at an appropriate level of detail. The tool can record the low level NFS calls themselves or an approximation of the user-level system calls (open, close, etc.) that triggered the activity. We partition the problem of deriving NFS activity from raw network traffic into two fairly distinct subprob lems: that of decoding the low-level NFS operations from the packets on the network, and that of translating these low-level commands back into user-level system calls. Hence, the toolkit consists of two basic parts, an "RPC decoder" (rpcspy) and the "NFS analyzer" (nfstrace). rpcspy communicates with a low-level network monitoring facility (such as Sun's NIT [9] or the Packetfilter [2]) to read and reconstruct the RPC transactions (call and reply) that make up each NFS command. nfstrace takes the output of rpcspy and reconstructs the sys tem calls that occurred as well as other interesting data it can derive about the structure of the filesystem, such as the mappings between NFS file handles and Unix file names. Since there is not a clean one-to-one mapping between system calls and lower-level NFS commands, nfstrace uses some simple heuristics to guess a reasonable approximation of what really occurred. 1.1. A Spy's View of the NFS Protocols It is well beyond the scope of this paper to describe the protocols used by NFS; for a detailed description of how NFS works, the reader is referred to [10], [11], and [12]. What follows is a very brief overview of how NFS activity translates into Ethernet packets. An NFS network consists of servers, to which filesystems are physically connected, and clients, which per form operations on remote server filesystems as if the disks were locally connected. A particular machine can be a client or a server or both. Clients mount remote server filesystems in their local hierarchy just as they do local filesystems; from the user's perspective, files on NFS and local filesystems are (for the most part) indistinguishable, and can be manipulated with the usual filesystem calls. The interface between client and server is defined in terms of 17 remote procedure call (RPC) operations. Remote files (and directories) are referred to by a file handle that uniquely identifies the file to the server. There are operations to read and write bytes of a file (read, write), obtain a file's attributes (getattr), obtain the contents of directories (lookup, readdir), create files (create), and so forth. While most of these operations are direct analogs of Unix system calls, notably absent are open and close operations; no client state information is maintained at the server, so there is no need to inform the server explicitly when a file is in use. Clients can maintain buffer cache entries for NFS files, but must verify that the blocks are still valid (by checking the last write time with the getattr operation) before using the cached data. An RPC transaction consists of a call message (with arguments) from the client to the server and a reply mes sage (with return data) from the server to the client. NFS RPC calls are transmitted using the UDP/IP connection less unreliable datagram protocol[13]. The call message contains a unique transaction identifier which is included in the reply message to enable the client to match the reply with its call. The data in both messages is encoded in an "external data representation" (XDR), which provides a machine-independent standard for byte order, etc. Note that the NFS server maintains no state information about its clients, and knows nothing about the context of each operation outside of the arguments to the operation itself. 2. The rpcspy Program rpcspy is the interface to the system-dependent Ethernet monitoring facility; it produces a trace of the RPC calls issued between a given set of clients and servers. At present, there are versions of rpcspy for a number of BSD-derived systems, including ULTRIX (with the Packetfilter[2]), SunOS (with NIT[9]), and the IBM RT running AOS (with the Stanford enet filter). For each RPC transaction monitored, rpcspy produces an ASCII record containing a timestamp, the name of the server, the client, the length of time the command took to execute, the name of the RPC command executed, and the command- specific arguments and return data. Currently, rpcspy understands and can decode the 17 NFS RPC commands, and there are hooks to allow other RPC services (for example, NIS) to be added reasonably easily. The output may be read directly or piped into another program (such as nfstrace) for further analysis; the for mat is designed to be reasonably friendly to both the human reader and other programs (such as nfstrace or awk). Since each RPC transaction consists of two messages, a call and a reply, rpcspy waits until it receives both these components and emits a single record for the entire transaction. The basic output format is 8 vertical-bar separated fields: timestamp | execution-time | server | client | command-name | arguments | reply-data where timestamp is the time the reply message was received, execution-time is the time (in microseconds) that elapsed between the call and reply, server is the name (or IP address) of the server, client is the name (or IP address) of the client followed by the userid that issued the command, command-name is the name of the particular program invoked (read, write, getattr, etc.), and arguments and reply-data are the command dependent arguments and return values passed to and from the RPC program, respectively. The exact format of the argument and reply data is dependent on the specific command issued and the level of detail the user wants logged. For example, a typical NFS command is recorded as follows: 690529992.167140 | 11717 | paramount | merckx.321 | read | {"7b1f00000000083c", 0, 8192} | ok, 1871 In this example, uid 321 at client "merckx" issued an NFS read command to server "paramount". The reply was issued at (Unix time) 690529992.167140 seconds; the call command occurred 11717 microseconds earlier. Three arguments are logged for the read call: the file handle from which to read (represented as a hexadecimal string), the offset from the beginning of the file, and the number of bytes to read. In this example, 8192 bytes are requested starting at the beginning (byte 0) of the file whose handle is "7b1f00000000083c". The command completed successfully (status "ok"), and 1871 bytes were returned. Of course, the reply message also included the 1871 bytes of data from the file, but that field of the reply is not logged by rpcspy. rpcspy has a number of configuration options to control which hosts and RPC commands are traced, which call and reply fields are printed, which Ethernet interfaces are tapped, how long to wait for reply messages, how long to run, etc. While its primary function is to provide input for the nfstrace program (see Section 3), judi cious use of these options (as well as such programs as grep, awk, etc.) permit its use as a simple NFS diag nostic and performance monitoring tool. A few screens of output give a surprisingly informative snapshot of current NFS activity; we have identified quickly using the program several problems that were otherwise difficult to pinpoint. Similarly, a short awk script can provide a breakdown of the most active clients, servers, and hosts over a sampled time period. 2.1. Implementation Issues The basic function of rpcspy is to monitor the network, extract those packets containing NFS data, and print the data in a useful format. Since each RPC transaction consists of a call and a reply, rpcspy maintains a table of pending call packets that are removed and emitted when the matching reply arrives. In normal operation on a reasonably fast workstation, this rarely requires more than about two megabytes of memory, even on a busy net work with unusually slow file servers. Should a server go down, however, the queue of pending call messages (which are never matched with a reply) can quickly become a memory hog; the user can specify a maximum size the table is allowed to reach before these "orphaned" calls are searched out and reclaimed. File handles pose special problems. While all NFS file handles are a fixed size, the number of significant bits varies from implementation to implementation; even within a vendor, two different releases of the same operating system might use a completely different internal handle format. In most Unix implementations, the handle contains a filesystem identifier and the inode number of the file; this is sometimes augmented by additional information, such as a version number. Since programs using rpcspy output generally will use the handle as a unique file identifier, it is important that there not appear to be more than one handle for the same file. Unfortunately, it is not sufficient to simply consider the handle as a bitstring of the maximum handle size, since many operating systems do not zero out the unused extra bits before assigning the handle. Fortunately, most servers are at least consistent in the sizes of the handles they assign. rpcspy allows the user to specify (on the command line or in a startup file) the handle size for each host to be monitored. The handles from that server are emitted as hexadecimal strings truncated at that length. If no size is specified, a guess is made based on a few common formats of a reasonable size. It is usually desirable to emit IP addresses of clients and servers as their symbolic host names. An early ver sion of the software simply did a nameserver lookup each time this was necessary; this quickly flooded the network with a nameserver request for each NFS transaction. The current version maintains a cache of host names; this requires a only a modest amount of memory for typical networks of less than a few hundred hosts. For very large networks or those where NFS service is provided to a large number of remote hosts, this could still be a potential problem, but as a last resort remote name resolution could be disabled or rpcspy configured to not translate IP addresses. UDP/IP datagrams may be fragmented among several packets if the datagram is larger than the maximum size of a single Ethernet frame. rpcspy looks only at the first fragment; in practice, fragmentation occurs only for the data fields of NFS read and write transactions, which are ignored anyway. 3. nfstrace: The Filesystem Tracing Package Although rpcspy provides a trace of the low-level NFS commands, it is not, in and of itself, sufficient for obtaining useful filesystem traces. The low-level commands do not by themselves reveal user-level activity. Furth ermore, the volume of data that would need to be recorded is potentially enormous, on the order of megabytes per hour. More useful would be an abstraction of the user-level system calls underlying the NFS activity. nfstrace is a filter for rpcspy that produces a log of a plausible set of user level filesystem commands that could have triggered the monitored activity. A record is produced each time a file is opened, giving a summary of what occurred. This summary is detailed enough for analysis or for use as input to a filesystem simulator. The output format of nfstrace consists of 7 fields: timestamp | command-time | direction | file-id | client | transferred | size where timestamp is the time the open occurred, command-time is the length of time between open and close, direc tion is either read or write (mkdir and readdir count as write and read, respectively). file-id identifies the server and the file handle, client is the client and user that performed the open, transferred is the number of bytes of the file actually read or written (cache hits have a 0 in this field), and size is the size of the file (in bytes). An example record might be as follows: 690691919.593442 | 17734 | read | basso:7b1f00000000400f | frejus.321 | 0 | 24576 Here, userid 321 at client frejus read file 7b1f00000000400f on server basso. The file is 24576 bytes long and was able to be read from the client cache. The command started at Unix time 690691919.593442 and took 17734 microseconds at the server to execute. Since it is sometimes useful to know the name corresponding to the handle and the mode information for each file, nfstrace optionally produces a map of file handles to file names and modes. When enough information (from lookup and readdir commands) is received, new names are added. Names can change over time (as files are deleted and renamed), so the times each mapping can be considered valid is recorded as well. The mapping infor mation may not always be complete, however, depending on how much activity has already been observed. Also, hard links can confuse the name mapping, and it is not always possible to determine which of several possible names a file was opened under. What nfstrace produces is only an approximation of the underlying user activity. Since there are no NFS open or close commands, the program must guess when these system calls occur. It does this by taking advantage of the observation that NFS is fairly consistent in what it does when a file is opened. If the file is in the local buffer cache, a getattr call is made on the file to verify that it has not changed since the file was cached. Otherwise, the actual bytes of the file are fetched as they are read by the user. (It is possible that part of the file is in the cache and part is not, in which case the getattr is performed and only the missing pieces are fetched. This occurs most often when a demand-paged executable is loaded). nfstrace assumes that any sequence of NFS read calls on the same file issued by the same user at the same client is part of a single open for read. The close is assumed to have taken place when the last read in the sequence completes. The end of a read sequence is detected when the same client reads the beginning of the file again or when a timeout with no reading has elapsed. Writes are handled in a similar manner. Reads that are entirely from the client cache are a bit harder; not every getattr command is caused by a cache read, and a few cache reads take place without a getattr. A user level stat system call can sometimes trigger a getattr, as can an ls -l command. Fortunately, the attribute caching used by most implementations of NFS seems to eliminate many of these extraneous getattrs, and ls commands appear to trigger a lookup command most of the time. nfstrace assumes that a getattr on any file that the client has read within the past few hours represents a cache read, otherwise it is ignored. This simple heuristic seems to be fairly accurate in practice. Note also that a getattr might not be performed if a read occurs very soon after the last read, but the time threshold is generally short enough that this is rarely a problem. Still, the cached reads that nfstrace reports are, at best, an estimate (generally erring on the side of over-reporting). There is no way to determine the number of bytes actually read for cache hits. The output of nfstrace is necessarily produced out of chronological order, but may be sorted easily by a post-processor. nfstrace has a host of options to control the level of detail of the trace, the lengths of the timeouts, and so on. To facilitate the production of very long traces, the output can be flushed and checkpointed at a specified inter val, and can be automatically compressed. 4. Using rpcspy and nfstrace for Filesystem Tracing Clearly, nfstrace is not suitable for producing highly accurate traces; cache hits are only estimated, the timing information is imprecise, and data from lost (and duplicated) network packets are not accounted for. When such a highly accurate trace is required, other approaches, such as modification of the client and server kernels, must be employed. The main virtue of the passive-monitoring approach lies in its simplicity. In [5], Baker, et al, describe a trace of a distributed filesystem which involved low-level modification of several different operating system kernels. In contrast, our entire filesystem trace package consists of less than 5000 lines of code written by a single programmer in a few weeks, involves no kernel modifications, and can be installed to monitor multiple heterogeneous servers and clients with no knowledge of even what operating systems they are running. The most important parameter affecting the accuracy of the traces is the ability of the machine on which rpcspy is running to keep up with the network traffic. Although most modern RISC workstations with reasonable Ethernet interfaces are able to keep up with typical network loads, it is important to determine how much informa tion was lost due to packet buffer overruns before relying upon the trace data. It is also important that the trace be, indeed, non-intrusive. It quickly became obvious, for example, that logging the traffic to an NFS filesystem can be problematic. Another parameter affecting the usefulness of the traces is the validity of the heuristics used to translate from RPC calls into user-level system calls. To test this, a shell script was written that performed ls -l, touch, cp and wc commands randomly in a small directory hierarchy, keeping a record of which files were touched and read and at what time. After several hours, nfstrace was able to detect 100% of the writes, 100% of the uncached reads, and 99.4% of the cached reads. Cached reads were over-reported by 11%, even though ls com mands (which cause the "phantom" reads) made up 50% of the test activity. While this test provides encouraging evidence of the accuracy of the traces, it is not by itself conclusive, since the particular workload being monitored may fool nfstrace in unanticipated ways. As in any research where data are collected about the behavior of human subjects, the privacy of the individu als observed is a concern. Although the contents of files are not logged by the toolkit, it is still possible to learn something about individual users from examining what files they read and write. At a minimum, the users of a mon itored system should be informed of the nature of the trace and the uses to which it will be put. In some cases, it may be necessary to disable the name translation from nfstrace when the data are being provided to others. Commercial sites where filenames might reveal something about proprietary projects can be particularly sensitive to such concerns. 5. A Trace of Filesystem Activity in the Princeton C.S. Department A previous paper[14] analyzed a five-day long trace of filesystem activity conducted on 112 research worksta tions at DEC-SRC. The paper identified a number of file access properties that affect filesystem caching perfor mance; it is difficult, however, to know whether these properties were unique artifacts of that particular environment or are more generally applicable. To help answer that question, it is necessary to look at similar traces from other computing environments. It was relatively easy to use rpcspy and nfstrace to conduct a week long trace of filesystem activity in the Princeton University Computer Science Department. The departmental computing facility serves a community of approximately 250 users, of which about 65% are researchers (faculty, graduate students, undergraduate researchers, postdoctoral staff, etc), 5% office staff, 2% systems staff, and the rest guests and other "external" users. About 115 of the users work full-time in the building and use the system heavily for electronic mail, netnews, and other such communication services as well as other computer science research oriented tasks (editing, compiling, and executing programs, formatting documents, etc). The computing facility consists of a central Auspex file server (fs) (to which users do not ordinarily log in directly), four DEC 5000/200s (elan, hart, atomic and dynamic) used as shared cycle servers, and an assortment of dedicated workstations (NeXT machines, Sun workstations, IBM-RTs, Iris workstations, etc.) in indi vidual offices and laboratories. Most users log in to one of the four cycle servers via X window terminals located in offices; the terminals are divided evenly among the four servers. There are a number of Ethernets throughout the building. The central file server is connected to a "machine room network" to which no user terminals are directly connected; traffic to the file server from outside the machine room is gatewayed via a Cisco router. Each of the four cycle servers has a local /, /bin and /tmp filesystem; other filesystems, including /usr, /usr/local, and users' home directories are NFS mounted from fs. Mail sent from local machines is delivered locally to the (shared) fs:/usr/spool/mail; mail from outside is delivered directly on fs. The trace was conducted by connecting a dedicated DEC 5000/200 with a local disk to the machine room net work. This network carries NFS traffic for all home directory access and access to all non-local cycle-server files (including the most of the actively-used programs). On a typical weekday, about 8 million packets are transmitted over this network. nfstrace was configured to record opens for read and write (but not directory accesses or individual reads or writes). After one week (wednesday to wednesday), 342,530 opens for read and 125,542 opens for write were recorded, occupying 8 MB of (compressed) disk space. Most of this traffic was from the four cycle servers. No attempt was made to "normalize" the workload during the trace period. Although users were notified that file accesses were being recorded, and provided an opportunity to ask to be excluded from the data collection, most users seemed to simply continue with their normal work. Similarly, no correction is made for any anomalous user activity that may have occurred during the trace. 5.1. The Workload Over Time Intuitively, the volume of traffic can be expected to vary with the time of day. Figure 1 shows the number of reads and writes per hour over the seven days of the trace; in particular, the volume of write traffic seems to mirror the general level of departmental activity fairly closely. An important metric of NFS performance is the client buffer cache hit rate. Each of the four cycle servers allocates approximately 6MB of memory for the buffer cache. The (estimated) aggregate hit rate (percentage of reads served by client caches) as seen at the file server was surprisingly low: 22.2% over the entire week. In any given hour, the hit rate never exceeded 40%. Figure 2 plots (actual) server reads and (estimated) cache hits per hour over the trace week; observe that the hit rate is at its worst during periods of the heaviest read activity. Past studies have predicted much higher hit rates than the aggregate observed here. It is probable that since most of the traffic is generated by the shared cycle servers, the low hit rate can be attributed to the large number of users competing for cache space. In fact, the hit rate was observed to be much higher on the single-user worksta tions monitored in the study, averaging above 52% overall. This suggests, somewhat counter-intuitively, that if more computers were added to the network (such that each user had a private workstation), the server load would decrease considerably. Figure 3 shows the actual cache misses and estimated cache hits for a typical private works tation in the study. Thu 00:00 Thu 06:00 Thu 12:00 Thu 18:00 Fri 00:00 Fri 06:00 Fri 12:00 Fri 18:00 Sat 00:00 Sat 06:00 Sat 12:00 Sat 18:00 Sun 00:00 Sun 06:00 Sun 12:00 Sun 18:00 Mon 00:00 Mon 06:00 Mon 12:00 Mon 18:00 Tue 00:00 Tue 06:00 Tue 12:00 Tue 18:00 Wed 00:00 Wed 06:00 Wed 12:00 Wed 18:00 1000 2000 3000 4000 5000 6000 Reads/Writes per hour Writes Reads (all) Figure 1 - Read and Write Traffic Over Time 5.2. File Sharing One property observed in the DEC-SRC trace is the tendency of files that are used by multiple workstations to make up a significant proportion of read traffic but a very small proportion of write traffic. This has important implications for a caching strategy, since, when it is true, files that are cached at many places very rarely need to be invalidated. Although the Princeton computing facility does not have a single workstation per user, a similar metric is the degree to which files read by more than one user are read and written. In this respect, the Princeton trace is very similar to the DEC-SRC trace. Files read by more than one user make up more than 60% of read traffic, but less than 2% of write traffic. Files shared by more than ten users make up less than .2% of write traffic but still more than 30% of read traffic. Figure 3 plots the number of users who have previously read each file against the number of reads and writes. 5.3. File "Entropy" Files in the DEC-SRC trace demonstrated a strong tendency to "become" read-only as they were read more and more often. That is, the probability that the next operation on a given file will overwrite the file drops off shar ply in proportion to the number of times it has been read in the past. Like the sharing property, this has implications for a caching strategy, since the probability that cached data is valid influences the choice of a validation scheme. Again, we find this property to be very strong in the Princeton trace. For any file access in the trace, the probability that it is a write is about 27%. If the file has already been read at least once since it was last written to, the write probability drops to 10%. Once the file has been read at least five times, the write probability drops below 1%. Fig ure 4 plots the observed write probability against the number of reads since the last write. Thu 00:00 Thu 06:00 Thu 12:00 Thu 18:00 Fri 00:00 Fri 06:00 Fri 12:00 Fri 18:00 Sat 00:00 Sat 06:00 Sat 12:00 Sat 18:00 Sun 00:00 Sun 06:00 Sun 12:00 Sun 18:00 Mon 00:00 Mon 06:00 Mon 12:00 Mon 18:00 Tue 00:00 Tue 06:00 Tue 12:00 Tue 18:00 Wed 00:00 Wed 06:00 Wed 12:00 Wed 18:00 1000 2000 3000 4000 5000 Total reads per hour Cache Hits (estimated) Cache Misses (actual) Figure 2 - Cache Hits and Misses Over Time 6. Conclusions Although filesystem traces are a useful tool for the analysis of current and proposed systems, the difficulty of collecting meaningful trace data makes such traces difficult to obtain. The performance degradation introduced by the trace software and the volume of raw data generated makes traces over long time periods and outside of comput ing research facilities particularly hard to conduct. Although not as accurate as direct, kernel-based tracing, a passive network monitor such as the one described in this paper can permit tracing of distributed systems relatively easily. The ability to limit the data collected to a high-level log of only the data required can make it practical to conduct traces over several months. Such a long term trace is presently being conducted at Princeton as part of the author's research on filesystem caching. The non-intrusive nature of the data collection makes traces possible at facilities where kernel modification is impracti cal or unacceptable. It is the author's hope that other sites (particularly those not doing computing research) will make use of this toolkit and will make the traces available to filesystem researchers. 7. Availability The toolkit, consisting of rpcspy, nfstrace, and several support scripts, currently runs under several BSD-derived platforms, including ULTRIX 4.x, SunOS 4.x, and IBM-RT/AOS. It is available for anonymous ftp over the Internet from samadams.princeton.edu, in the compressed tar file nfstrace/nfstrace.tar.Z. Thu 00:00 Thu 06:00 Thu 12:00 Thu 18:00 Fri 00:00 Fri 06:00 Fri 12:00 Fri 18:00 Sat 00:00 Sat 06:00 Sat 12:00 Sat 18:00 Sun 00:00 Sun 06:00 Sun 12:00 Sun 18:00 Mon 00:00 Mon 06:00 Mon 12:00 Mon 18:00 Tue 00:00 Tue 06:00 Tue 12:00 Tue 18:00 Wed 00:00 Wed 06:00 Wed 12:00 Wed 18:00 0 100 200 300 Reads per hour Cache Hits (estimated) Cache Misses (actual) Figure 3 - Cache Hits and Misses Over Time - Private Workstation 0 5 10 15 20 n (readers) 0 20 40 60 80 100 % of Reads and Writes used by > n users Reads Writes Figure 4 - Degree of Sharing for Reads and Writes 0 5 10 15 20 Reads Since Last Write 0.0 0.1 0.2 P(next operation is write) Figure 5 - Probability of Write Given >= n Previous Reads 8. Acknowledgments The author would like to gratefully acknowledge Jim Roberts and Steve Beck for their help in getting the trace machine up and running, Rafael Alonso for his helpful comments and direction, and the members of the pro gram committee for their valuable suggestions. Jim Plank deserves special thanks for writing jgraph, the software which produced the figures in this paper. 9. References [1] Sandberg, R., Goldberg, D., Kleiman, S., Walsh, D., & Lyon, B. "Design and Implementation of the Sun Net work File System." Proc. USENIX, Summer, 1985. [2] Mogul, J., Rashid, R., & Accetta, M. "The Packet Filter: An Efficient Mechanism for User-Level Network Code." Proc. 11th ACM Symp. on Operating Systems Principles, 1987. [3] Ousterhout J., et al. "A Trace-Driven Analysis of the Unix 4.2 BSD File System." Proc. 10th ACM Symp. on Operating Systems Principles, 1985. [4] Floyd, R. "Short-Term File Reference Patterns in a UNIX Environment," TR-177 Dept. Comp. Sci, U. of Rochester, 1986. [5] Baker, M. et al. "Measurements of a Distributed File System," Proc. 13th ACM Symp. on Operating Systems Principles, 1991. [6] Metcalfe, R. & Boggs, D. "Ethernet: Distributed Packet Switching for Local Computer Networks," CACM July, 1976. [7] "Etherfind(8) Manual Page," SunOS Reference Manual, Sun Microsystems, 1988. [8] Gusella, R. "Analysis of Diskless Workstation Traffic on an Ethernet," TR-UCB/CSD-87/379, University Of California, Berkeley, 1987. [9] "NIT(4) Manual Page," SunOS Reference Manual, Sun Microsystems, 1988. [10] "XDR Protocol Specification," Networking on the Sun Workstation, Sun Microsystems, 1986. [11] "RPC Protocol Specification," Networking on the Sun Workstation, Sun Microsystems, 1986. [12] "NFS Protocol Specification," Networking on the Sun Workstation, Sun Microsystems, 1986. [13] Postel, J. "User Datagram Protocol," RFC 768, Network Information Center, 1980. [14] Blaze, M., and Alonso, R., "Long-Term Caching Strategies for Very Large Distributed File Systems," Proc. Summer 1991 USENIX, 1991. Matt Blaze is a Ph.D. candidate in Computer Science at Princeton University, where he expects to receive his degree in the Spring of 1992. His research interests include distributed systems, operating systems, databases, and programming environments. His current research focuses on caching in very large distributed filesys tems. In 1988 he received an M.S. in Computer Science from Columbia University and in 1986 a B.S. from Hunter College. He can be reached via email at mab@cs.princeton.edu or via US mail at Dept. of Computer Science, Princeton University, 35 Olden Street, Princeton NJ 08544.