An integrated multimodal feature fusion with adaptive and attention residual deep learning network for personalized learning recommendation

Abstract

In this research work, a novel multimodal data-based recommendation system is introduced to ultimately enhance student's talents and broaden knowledge without any interference. Initially, the required multimodal data, such as video, images, and text data are gathered from standard sources and it is given to the feature extraction phase. Here, the Bidirectional Encoder Representations from Transformers (BERT), Visual Geometry Group 16-based Autoencoder (VGG16-AE) and 3D VGG16-AE mechanisms are utilized for accurately recognizing and differentiating the significant feature sets such as, first set of features (f1), second set of features (f2) and third set of features (f3) from the input data. Then, the extracted three set of features are fed into the learning recommendation phase here, the Multi-scale Feature Fusion-based Residual Bidirectional Gated recurrent unit with Sparse Attention (MFF-RBG-SA) mechanism is designed to efficiently analyze and generate better solution in an earlier stage. Moreover, the overall performance of the designed approach is estimated with various performance measures and compared with conventional methods to guarantee its reliable effectiveness in the educational sector. In the validation phase, the designed approach has attained higher accuracy value of 95.35%, 95.28%, 95.38% and 95.33% in terms of Linear, Tanh, softmax and sigmoid activation functions, which demonstrate its superior learning recommendation performance than other conventional methods.