Wolfgang Effelsberg
Abstract
This paper discusses the implementation of a database buffer manager as a component of a DBMS. The interface between calling components of higher system layers and the buffer manager is described; the principal differences between virtual memory paging and database buffer management are outlined; the notion of referencing versus addressing of database pages is introduced; and the concept of fixing pages in the buffer to prevent uncontrolled replacement is explained.
Three basic tasks have to be performed by the buffer manager: buffer search, allocation of frames to concurrent transactions, and page replacement. For each of these tasks, implementation alternatives are discussed and illustrated by examples from a performance evaluation project of a CODASYL DBMS.
- 1 BABAOGLU, O., AND JoY, W. Converting a swap-based system to do paging in an architecture lacking page-referenced bits. In Proceedings 8th Symposium on Operating Systems Principles. SIGOPS 15, 5 (Dec. 1981), 78-86. Google Scholar
Digital Library
- 2 BELADY, L.A. A study of replacement algorithms for virtual storage computers. IBM Syst. J. 5, 2 (1966), 78-101.Google ScholarDigital Library
- 3 BR{CE, R.S., AND SHERMAN, S.W. An extension of the performance of a database manager in a virtual memory system using partially locked virtual buffers. ACM Trans. Database Syst. 2, 2 (1977), 196-207. Google ScholarDigital Library
- 4 CARR, R.W., AND HENNESS~, J.L. WSCLOCK--a simple and effective algorithm for virtual memory management, in Proceedings 8th Symposium on Operating Systems Principles. SIGOPS 15, 5 (Dec. 1981), 87-95. Google ScholarDigital Library
- 5 CHU, W.W., AND OPDERBECK, H. Program behavior and the page fault frequency replacement algorithm. Computer 9, 11 (1976), 29-38.Google ScholarDigital Library
- 6 DENNING, P.J. The working set model for program behavior. Commun. ACM 11, 5 (1968), 323- 333. Google ScholarDigital Library
- 7 DENNING, P.J. Thrashing: Its causes and prevention. In AFIPS Conference Proceedings, Vol. 33, FJCC, 1968, 915-922.Google Scholar
- 8 DENNING, P.J. Working sets past and present. IEEE Trans. Softw. Eng. SE-6, 1 (1980), 64-84.Google ScholarDigital Library
- 9 EFFELSBERG, W. Buffer management in database systems. Dissertation, Fachbereich Informatik, Technische Hochschule Darmstadt, 1981, (in German).Google Scholar
- 10 FERNANDEZ, E.B., LANG, T., AND WOOD, C. Effect of replacement algorithms on a paged buffer database system. IBM J. Res. DeF. 22, 2 (1978), 185-196.Google ScholarDigital Library
- 11 FERRARI, D. The improvement of program behavior. Computer 9, 11 (1976), 39-47.Google Scholar
- 12 HA~RDER, T. Embedding a database system in an operating system environment. In Datenbanktechnologie, J. Niedereichholz, Ed. Proceedings II/79 of the German Chapter of the ACM, Teubner-Verlag, Stuttgart, 1979, 9-24, (in German).Google Scholar
- 13 HOWARD, J.H. Virtual memory buffering, IBM Res. Rep., San Jose, Calif., 1980, (in preparation).Google Scholar
- 14 LAN(}, T., WOOD, C., AND FERNANDEZ, E.B. Database buffer paging in virtual storage systems. ACM Trans. Database Syst. 2, 4 (1977), 339-351. Google ScholarDigital Library
- 15 MATTSON, R.L., GECSEX, J., SLUTZ, D.R,, AND TRAIGER, I.L. Evaluation techniques for storage hierarchies. IBM Syst. J. 9, 2 (1970), 78-117.Google ScholarDigital Library
- 16 REXTER, A. A study of buffer management policies for data management systems. Tech. Summary Rep. No. 1619, Mathematics Research Center, Univ. of Wisconsin, Madison, Mar. 1976.Google Scholar
- 17 RODRIGUEZ-ROSELL, J., AND DUPUY, J-P. The design, implementation, and evaluation of a working set dispatcher. Commun. ACM 16, 4 (1973), 247-253. Google ScholarDigital Library
- 18 RODRIGUEZ-ROSELL, J. Empirical data reference behavior in database systems. Computer 9, 11 (1976), 9-13.Google ScholarDigital Library
- 19 SACCO, G.M., AND SCHKOLNICK, M. A mechanism for managing the buffer pool in a relational database system using the hot set model. In Proceedings 8th Conference on Very Large Data Bases, (Mexico, 1982). Google ScholarDigital Library
- 20 SHERMAN, S.W., AND BRICE, R.S. Performance of a database manager in a virtual memory system. ACM Trans. Database Syst. I, 4 (1976), 317-343. Google ScholarDigital Library
- 21 SMITH, A.J. Sequentiality and prefetching in database systems. A CM Trans. Database Syst. 3, 3 (1978), 223-247. Google ScholarDigital Library
- 22 SPIRN, J. Distance string models for program behavior. Computer 9, 11 (1976), 14-20.Google ScholarDigital Library
- 23 SPIRN, J.R., AND DENNING, P.J. Experiments with program locality. In AFIPS Con/erence Proceedings, Vol. 41, FJCC, 1972, 611-621.Google Scholar
- 24 STONEBRAKER, M. Operating system support for database management systems. Commun. ACM 24, 7 (1981), 412-418. Google ScholarDigital Library
- 25 TUEL, W.G. An analysis of buffer paging in virtual storage systems. IBM J. Res. DeF. 20, 5 (1976), 518-520.Google ScholarDigital Library
- 26 TURNER, R., AND STRECKER, B. Use of the LRU stack depth distribution for simulation of paging behavior. Commun. ACM 20, 11 (1977), 795-798. Google ScholarDigital Library
Index Terms
- Principles of database buffer management
Recommendations
Second-Level Buffer Cache Management
Abstract--Buffer caches are commonly used in servers to reduce the number of slow disk accesses or network messages. These buffer caches form a multilevel buffer cache hierarchy. In such a hierarchy, second-level buffer caches have different access ...
Flash-Aware Buffer Management for Database Systems
Classical buffer replacement policies, e. g., LRU, are suboptimal for database systems having flash disks for persistence, because they are not aware of the distinguished characteristics of flash-based storage devices. The authors introduce the basic ...
Reviews
Mark Robbin Brown
The buffer manager of a DataBase Management System (DBMS) is the software component responsible for using a limited amount of main storage as disk page buffers, thereby reducing the number of disk I/Os per database transaction. This paper describes an interface (FIX-UNFIX) between the buffer manager and the rest of a DBMS, and discusses the principal choices facing the implementor of a buffer manager. It covers the possible page replacement policies (algorithms for choosing an existing page buffer to be mapped to a new disk page) in great detail. Perhaps the paper discusses design alternatives a bit too exhaustively. For instance, the possibility of doing buffer allocation on a per-transaction basis is discussed in great detail in Section 4.2. This is then dismissed in Section 6.1 by reasoning that does not depend on the intervening material. The paper does not do justice to the issues involved in optimizing sequential I/O. The possibility of read-ahead is only mentioned briefly, and write-behind (determining that several page writes are contiguous on the disk so they can be done in a batch) is not mentioned at all. For the buffer manager to apply these techniques successfully, the DBMS must supply it with some additional information when sequential accesses are taking place, but fortunately the DBMS often has this information. If, as the paper advocates, a global buffer pool is used, then one transaction performing sequential accesses can fill the buffer pool with worthless pages unless sequential I/O is treated specially. The paper discusses some interactions between a buffer manager and an operating system, for instance “double paging” (doing several physical I/Os to satisfy one logical I/O request in a virtual memory environment). But there are other interactions worth examining. At a very basic level, the design of a buffer manager (and the rest of a DBMS) is greatly influenced by the operating system's support for creating separate threads of control and communicating information between them. The authors of this paper seem to understand quite a number of existing database management systems; I wish they had examined the relationships between these systems and their host operating systems in more depth. The authors do not mention one of the more interesting recent works on the subject [1].
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments