Storing data from simulation runs for compactness and quick retrieval

Abstract

Simulation models can generate tremendous quantities of data. This paper describes data compression and file organization tech niques which were developed to reduce the disk space required by these data. The compression techniques, which include prefix encoding, differencing and scaling, reduced one file of state variable histories by 73%. Also described is a method which allows histories of many variables to share a single file without requiring individual recorded values to be identified as to their source. This technique reduced by 99% the time to extract the history of a single variable from a file of histories of 600 state variables.