Although there are several extant studies of parallel scien tific application request patterns, there is little experimen tal data on the correlation of physical I/O patterns with application I/O stimuli. To understand these correlations, the authors have instrumented the SCSI device drivers of the Intel Paragon OSF/1 operating system to record key physical I/O activities, and have correlated this data with the I/O patterns of scientific applications captured via the Pablo analysis toolkit. This analysis shows that disk hard ware features profoundly affect the distribution of request delays and that current parallel file systems respond to parallel application I/O patterns in nonscalable ways.
Get full access to this article
View all access options for this article.
References
1.
Baker, M.G.1991. Measurements of a distributed file system Proceedings of the Thirteenth Symposium on Operating System Principles 25. Association for Computing Machinery, pp. 198-212.
2.
Crandall, P., Aydt, R.A., Chien, A.A., and Reed, D.A.1996. I/O characterization of scalable parallel applications In Proceedings of Supercomputing 1995, San Diego.
3.
Forin, A., Golub, D., and Bershad, B.1991. An I/O system for Mach 3.0 In Proceedings of the USENIX Mach Symposium, USENIX, pp. 163-176.
4.
Gibson, G.A., Vitter, J.S., and Wilkes, J.1996. Strategic directions in computing research: Working group on storage I/O issues in large-scale computingACM Computing Surveys28(4): 779-793.
5.
High Performance Computational Chemistry Group.1995. NWChem, a computational chemistry package for parallel computers, version 1.1 Available at Pacific Northwest National Laboratory, Richland, WA, 99352, U.S.A..
6.
Jensen, D.W., and Reed, D.A.1993. File archive activity in a supercomputing environment In Proceedings of the 1993 ACM International Conference on Supercomputing, July.
7.
Kotz, D., and Nieuwejaar, N.1994. Dynamic file-access characteristics of a production parallel scientific workload In Proceedings of Supercomputing '94, Los Alamitos, November, pp. 640-649.
8.
Lawrie, D.H., Randal, J.M., and Barton, R.R.1982. Experiments with automatic file migrationIEEE Computer, July, pp. 45-55.
9.
Madhyastha, T., and Reed, D.A.1996. Intelligent, adaptive file system policy selection In Proceedings of Frontiers '96, October, 172-179.
10.
Miller, E.L., and Katz, R.H.1991. I/O behavior of supercomputer applications In Proceedings of Supercomputing '91, November, pp. 567-576.
11.
Pasquale, B.K., and Polyzos, G.A.1993, Static analysis of I/O characteristics of scientific applications in a production workload In Proceedings of Supercomputing '93, Portland, November, pp. 388-397.
12.
Pasquale, B.K., and Polyzos, G.C.1994. Dynamic I/O characterization of I/O intensive scientific applications In Proceedings of Supercomputing '94 , Washington, DC, November, pp. 660-669.
13.
Pool, J.T.1996. Scalable I/O InitiativeCalifornia Institute of Technology. Available at http://www.ccsf.caltech.edulSIO/
14.
Purakayastha, A., Ellis, C.S., Kotz, D., Nieuwejaar, N., and Best, M.1995. Characterizing parallel file access patterns on a large-scale multiprocessor In Proceedings of the Ninth International Parallel Processing Symposium, April, pp. 165-172.
15.
Reed, D.A., Aydt, R.A., Noe, R.J., Roth, P.C., Shields, K.A., Schwartz, B.W., and Tavera, L.F.1993. Scalable performance analysis: The Pablo performance analysis environment In Proceedings of the Scalable Parallel Libraries Conference, edited by A. Skjellum. Los Alamitos, CA: IEEE Computer Society Press, 1993, pp. 104-113.
16.
Reed, D.A., Elford, C.L., Madhyastha, T., Scullin, W.H., Aydt, R.A., and Smimi, E.1996. I/O, performance analysis, and performance data immersion In Proceedings of MASCOTS '96 , San Jose, February, pp. 1-12.
17.
Reed, D.A., Elford, C.L., Madhyastha, T., Smirni, E., and Lamm, S.L.1996. The next frontier: Interactive and closed loop performance steering In Proceedings of the 1996 International Conference on Parallel Processing Workshop, Bloomington, August, pp. 20-31.
18.
Ruemmler, C., and Wilkes, J.1993. UNIX disk access patterns In Proceedings of the Winter 1993 USENIX Conference, USENIX, pp. 405-420.
19.
Ruemmler, C., and Wilkes, J.1994. An introduction to disk drive modelingComputer27(3):17-28.
20.
Seltzer, M., Chen, P., and Ousterhout, J.1990. Disk scheduling revisited In Proceedings of the Winter 1990 USENIX Conference, January, USENIX, pp. 313-324.
21.
Smimi, E., Aydt, R.A., Chien, A.A., and Reed, D.A.1996. I/O requirements of scientific applications: An evolutionary viewHigh Performance Distributed Computing , pp. 49-59.
22.
Smirni, E., Aydt, R.A., Chien, A.A., and Reed, D.A.1996. I/O requirements of scientific applications: An evolutionary view In Proceedings of the Fifth IEEE International Symposium on High-Performance Distributed Computing, August, pp. 49-59.
23.
Smirni, E., and Reed, D.A.1997. Workload characterization of I/O intensive parallel applications In Proceedings of the 9th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation , June.
24.
Smith, A.J.1981. Analysis of long term file reference patterns for application to file migration algorithms IEEE Transactions on Software Engineering SE-7 4:403-417.
25.
Stritter, T.R.1977. File migration Ph.D. thesis, Department of Computer Science, Stanford University.
26.
Wilkes, J., Worthington, B.L., Ganger, G.R., and Patt, Y.N.1995. On-line extraction of SCSI disk drive parameters In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, Ottawa, Canada, May, pp. 146-156.
