On incremental collaborative appearance model and regional particle filtering for lip region tracking

Abstract

Lip region tracking is of crucial importance to the better understanding of visual speech in a computer-aided system. This paper presents an efficient lip region tracking approach in virtue of incremental collaborative appearance model and regional particle filtering. Within this approach, we first learn an incremental weighted appearance model (IWAM) through adaptively updating the time-varying mean and eigenbasis by considering the temporal and spatial weights, and then discriminatively exploit an incremental sparse subspace model (ISSM) by considering the occlusions and background clutters. Accordingly, the collaboration of the IWAM and ISSM leads to a more reliable and flexible lip region representation. Subsequently, we propose a regional particle filtering for motion state estimation, by taking scale size and rotation estimation, reducing the computational load and alleviating the tracking drift into consideration. Furthermore, the affinely warped image patches corresponding to the rank-2-optimal states are employed to incrementally update the IWAM and ISSM synchronously. The extensive experiments conducted on different challenging sequences have shown that the improvement from the proposed framework compared to the state-of-the-art systems, are over 15%, 18% and 20%, on reducing the errors of center location, scale size and rotation angle estimations, respectively.