Posts Tagged ‘File System

The CIFS Protocol

The CIFS (Common Internet File System) protocol, which is based on the Server Message Block (SMB) protocol that defines most of the file transfer operations used by CIFS, differs in a number of respects from oft-studied NFSv2 and v3. Most importantly, CIFS is stateful: CIFS user and file operations act on stateful session IDs and file handles, making analysis of access patterns simpler and more accurate than in NFSv2 and v3, which require heuristics to infer the start and end of an access. Although CIFS may differ from other protocols, we believe our observations are not tied exclusively to CIFS because access patterns, file sharing, and other workload characteristics observed by the file server are influenced by the file system users, their applications,and the behavior of the file system client, none of which are closely tied to the transfer protocol.

Tags : , , , , , , , ,

Storage Systems and the Grid Storage API

In a Grid environment, data may be stored in diff erent locations and on diff erent devices with di fferent characteristics. The mechanism neutrality implies that applications should not need to be aware of the speci c low-level mechanisms required to access data at a particular location. Instead, applications should be presented with a uniform view of data and with uniform mechanisms for accessing that data. These requirements are met by the storage system abstraction and our grid storage API. Together, these de ne our data access service.

1. Data Abstraction: Storage Systems

We introduce as a basic data grid component what we call a storage system, which we de ne as an entity that can be manipulated with a set of functions for creating, destroying, reading, writing, and manipulating the attributes of named sequences of bytes called fi le instances. Notice that our defi nition of a storage system is a logical one: a storage system can be implemented by any storage technology that can support the required access functions. Implementations that target Unix fi le systems, HTTP servers, hierarchical storage systems such as HPSS, and network caches such as the Distributed Parallel Storage System (DPSS) are certainly envisioned. In fact, a storage system need not map directly to a single low-level storage device. For example, a distributed file system that manages fi les distributed over multiple storage devices or even sites can serve as a storage system, as can an SRB system that serves requests by mapping to multiple storage systems of di fferent types.

Our defi nition of a fi le instance is also logical rather than physical. A storage system holds data, which may actually be stored in a fi le system, database, or other system; we do not care about how data is stored but specify simply that the basic unit that we deal with is a named sequences of uninterpreted bytes. The use of the term “fi le instance” for this basic unit is not intended to imply that the data must live in a conventional fi le system. For example, a data grid implementation might use a system such as SRB to access data stored within a database management system. A storage system will associate with each of the le instances that it contains a set of properties, including a name and attributes such as its size and access restrictions. The name assigned to a leinstance by a particular storage system is arbitrary and has meaning only to that storage system. In many storage systems, a name will be a hierarchical directory path. In other systems such as SRB, it may be a set of application metadata that the storage system maps internally to a physical leinstance.

2. Grid Storage API

The behavior of a storage system as seen by a data grid user is defi ned by the data grid storage API, which de fines a variety of operations on storage systems and fi le instances. Our understanding of the functionality required in this API is still evolving, but it certainly should include support for remote requests to read and/or write named fi le instances and to determine fi le instance attributes such as size. In addition, to support optimized implementation of replica management services (discussed below) we require a third party transfer operation used to transfer the entire contents of a le instance from one storage system to another.

While the basic storage system functions just listed are relatively simple, various data grid considerations can increase the complexity of an implementation. For example, storage system access functions must be integrated with the security environment of each site to which remote access is required. Robust performance within higher-level functions requires reservation capabilities within storage systems and network interfaces. Applications should be able to provide storage systems with hints concerning access patterns, network performance, and so forth that the storage system can use to optimize its behavior. Similarly, storage systems should be capable of characterizing and monitoring their own performance; this information, when made available to storage system clients, allows them to optimize their behavior. Finally, data movement functions must be able to detect and report errors. While it may be possible to recover from some errors with in the storage system, other errors may need to reported back to the remote application that initiated the movement.

Tags : , , , , , , , , , , , , , , , , , ,

Performance Analysis of Cache Policies for Web Servers

Many existing Web servers, e.g., NCSA and Apache, rely on the underlying file system buffer of the operating system to cache recently accessed documents. When a new request arrives, the Web server asksthe operating system to open the file containing the requested document and starts reading it into a temporary memory buffer. After the file has been read, it needs to be closed.

Web Server Caching vs. File System and Database Caching

Traditional file system caches do not perform well for the WWW load [A+95, M96]. The following three differences between the traditional and Web caching account for that:

1. Web data items have a different granularity.File system and database buffers deal with fixed size blocks of data. Web servers always read andcache entire files.  Additionally,non-fixed size data items complicate memory management in Web server caches.

2. Caching is not obligatory in Web servers, i.e., some documents may not be admitted in the cache.A file system/database buffer manager always places requested data blocks in the cache (the cache serves as an interface between the storage subsystem and the application). On the contrary, a Webserver cache manager may choose not to buffer a document if this can increase cache performance(hit rate). This option of not caching some documents combined with a different granularity of data items significantly broadens the variety of cache management policies that can be used in a Webserver.

3. There are no correlated re-reads or sequential scans in Web workloads.Unlike database systems, Web servers never experience sequential scans of a large number of data items. Nor do they have correlated data re-reads. That is, the same user never reads a document twice in a short time interval. This becomes obvious if we take the Web browser cache into account. All documents re-reads are handled there. The Web server never knows about them. (Communicationfailures, dynamically generated documents, and browser cache malfunctioning may result incorrelated re-reads on the Web server).

One of the reasons why correlated rereads are common in traditional caches is the possibility of storing several logical data items, e.g., database records, in the same physical block. As a result, even accesses to different logical records may touch the same buffer page, e.g., during a sequential scan,resulting in artificial page re-reads.

The impact of the absence of correlated re-reads is two-fold. Firstly, a caching algorithm for Web workloads does not need to “factor out locality” [RV90], i.e. eliminate the negative impact of correlated re-reads on cache performance. Repeated (correlated) accesses to a data item from a single application make the cache manager consider the item popular even though this popularity is artificial: all accesses occur in a short time interval and the item is rarely used. In Web servers, on the contrary, multiple accesses in a short time interval do indicate high popularity of a document since the accesses came from different sources.

Secondly, a single access to a document should not be a reason for the cache manager to put the document in the cache since there will be no following, correlated accesses. In other words, the traditional LRU (Least Recently Used) is not suitable for Web document caching.

Additionally, Web servers deal with fewer data items than file systems and databases do. The number of documents stored on a typical Web server rarely exceeds 100,000 whereas files systems and databases have to deal with millions of blocks. As a result, a Web server can afford keeping access statistics for all stored documents while traditional systems cannot.

Keeping the foregoing observations in mind, one can expect that a dedicated document cache would perform better than a file system cache. Such a document cache should use a cache management policy that is more suitable for the WWW load than that of the operating system.


Tags : , , , , ,