Development of an enhanced transformer framework for industrial virtual metrology on dynamic processes

Abstract

In modern industry, it is of great significance to employ data-driven virtual metrology technique to improve production efficiency and quality by predicting key quality variables in an economical and reliable way. Among data-driven algorithms, the transformer is yielding promising results in predicting time series data and handling vast amounts of complex industrial data, due to the superior attention mechanism. In this paper, a novel target adaptive attention (TAA) mechanism is first developed for guiding transformer to focus more on characteristics relating quality variables, by ensuring that the encoder would adaptively identify and capture features with higher correlation with the target quality variables. To handle large-scale industrial data while both the dimensionality and scale of data expanding, the number of encoder layers would increase accordingly, such that the predictive performance of the model would decline; thus successively, the inter-level fusion attention (IFA) mechanism is proposed to add the weighted evaluation of interlevel correlations among different encoder layers to transformer for improving capabilities of feature extraction and enhancing prediction accuracy. Experiments on virtual metrology tasks on the DC process and the primary reformer process illustrate the merits of the proposed method in a sense that the target quality variable is accurately predicted.