Decoding of human brain activity has always been a primary goal in neuroscience especially with functional magnetic resonance imaging (fMRI) data. In recent years, Convolutional neural network (CNN) has become a popular method for the extraction of features due to its higher accuracy, however it needs a lot of computation and training data. In this study, an algorithm is developed using Multivariate pattern analysis (MVPA) and modified CNN to decode the behavior of the brain for different images with limited data sets. Selection of significant features is an important part of fMRI data analysis, since it reduces the computational burden and improves the prediction performance; significant features are selected using a t-test. MVPA uses machine learning algorithms to classify different brain states and helps with prediction during the task. General linear model (GLM) is used to find the unknown parameters of every individual voxel and a classification is done using a multi-class support vector machine (SVM). A proposed MVPA-CNN based algorithm is compared with a region of interest (ROI) based method and MVPA based estimated values. The proposed method showed better overall accuracy (68.6%) compared to ROI (61.88%) and estimation values (64.17%).
CecottiH. and GraserA., Convolutional neural networks for P300 detection with application to brain-computer interfaces, IEEE transactions on pattern analysis and machine intelligence33 (2011), 433–445. doi:10.1109/TPAMI.2010.125.
3.
ChouC.-A., KampaK., MehtaS.H., TungarazaR.F., ChaovalitwongseW.A. and GrabowskiT.J., Voxel selection framework in multi-voxel pattern analysis of fMRI data for prediction of neural response to visual stimuli, Medical Imaging, IEEE Transactions on33 (2014), 925–934. doi:10.1109/TMI.2014.2298856.
FriedmanJ., Another approach to polychotomous classification, Technical report, Department of Statistics, Stanford University, 1996.
6.
HajinorooziM., JungT.-P., LinC.-T. and HuangY., Feature extraction with deep belief networks for driver's cognitive states prediction from EEG data, in: Signal and Information Processing (ChinaSIP), 2015 IEEE China Summit and International Conference on, 2015, pp. 812–815. doi:10.1109/ChinaSIP.2015.7230517.
7.
HaynesJ.-D. and ReesG., Decoding mental states from brain activity in humans, Nature Reviews Neuroscience7 (2006), 523–534. doi:10.1038/nrn1931.
8.
KayK.N., NaselarisT., PrengerR.J. and GallantJ.L., Identifying natural images from human brain activity, Nature452 (2008), 352–355. doi:10.1038/nature06713.
9.
KnerrS., PersonnazL. and DreyfusG., Single-Layer Learning Revisited: A Stepwise Procedure for Building and Training a Neural Network, Neurocomputing, Springer, 1990, pp. 41–50.
10.
KreßelU.H.-G., Pairwise classification and support vector machines, in: Advances in Kernel Methods, 1999, pp. 255–268.
11.
KriegeskorteN., Deep neural networks: A new framework for modeling biological vision and brain information processing, Annual Review of Vision Science1 (2015), 417–446. doi:10.1146/annurev-vision-082114-035447.
12.
LeCunY., BottouL., BengioY. and HaffnerP., Gradient-based learning applied to document recognition, Proceedings of the IEEE86 (1998), 2278–2324. doi:10.1109/5.726791.
13.
LeCunY., HuangF.J. and BottouL., Learning methods for generic object recognition with invariance to pose and lighting, in: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, 2004, pp. 97–104.
14.
McMillanD.E., Increased levels of acute-phase serum proteins in diabetes, Metabolism38 (1989), 1042–1046. doi:10.1016/0026-0495(89)90038-3.
15.
MirowskiP.W., LeCunY., MadhavanD. and KuznieckyR., Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial EEG, in: Machine Learning for Signal Processing, 2008, MLSP 2008. IEEE Workshop on, 2008, pp. 244–249.
16.
Mourao-MirandaJ., BokdeA.L., BornC., HampelH. and StetterM., Classifying brain states and determining the discriminating activation patterns: Support vector machine on functional MRI data, NeuroImage28 (2005), 980–995. doi:10.1016/j.neuroimage.2005.06.070.
17.
MumfordJ.A., TurnerB.O., AshbyF.G. and PoldrackR.A., Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses, NeuroImage59 (2012), 2636–2643. doi:10.1016/j.neuroimage.2011.08.076.
18.
MumfordJ.A., DavisT. and PoldrackR.A., The impact of study design on pattern estimation for single-trial multivariate pattern analysis, NeuroImage103 (2014), 130–138. doi:10.1016/j.neuroimage.2014.09.026.
19.
NagiJ., Di CaroG.A., GiustiA., NagiF. and GambardellaL.M., Convolutional neural support vector machines: Hybrid visual pattern classifiers for multi-robot systems, in: Machine Learning and Applications (ICMLA), 2012 11th International Conference on, 2012, pp. 27–32. doi:10.1109/ICMLA.2012.14.
20.
NaselarisT., PrengerR.J., KayK.N., OliverM. and GallantJ.L., Bayesian reconstruction of natural images from human brain activity, Neuron63 (2009), 902. doi:10.1016/j.neuron.2009.09.006.
21.
NormanK.A., PolynS.M., DetreG.J. and HaxbyJ.V., Beyond mind-reading: Multi-voxel pattern analysis of fMRI data, Trends in Cognitive Sciences10(9) (2006), 424–430.
22.
SchoenmakersS., BarthM., HeskesT. and van GervenM., Linear reconstruction of perceived images from human brain activity, NeuroImage83 (2013), 951–961. doi:10.1016/j.neuroimage.2013.07.043.
23.
SchölkopfB. and SmolaA.J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, 2002.
24.
Shawe-TaylorJ. and CristianiniN., Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.
25.
SimardP.Y., SteinkrausD. and PlattJ.C., Best practices for convolutional neural networks applied to visual document analysis, in: ICDAR, 2003, pp. 958–962.
26.
StoberS., CameronD.J. and GrahnJ.A., Using convolutional neural networks to recognize rhythm stimuli from electroencephalography recordings, in: Advances in Neural Information Processing Systems, 2014, pp. 1449–1457.
27.
TangY., Deep learning using linear support vector machines, arXiv:1306.0239, 2013.
28.
ThirionB., DuchesnayE., HubbardE., DuboisJ., PolineJ.-B., LebihanD. and DehaeneS., Inverse retinotopy: Inferring the visual content of images from brain activation patterns, NeuroImage33 (2006), 1104–1116. doi:10.1016/j.neuroimage.2006.06.062.
29.
TongF. and PratteM.S., Decoding patterns of human brain activity, Annual review of psychology63 (2012), 483–509. doi:10.1146/annurev-psych-120710-100412.
30.
WangL., ChuF. and XieW., Accurate cancer classification using expressions of very few genes, IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)4 (2007), 40–53. doi:10.1109/TCBB.2007.1006.
31.
WangL. and FuX., Data Mining with Computational Intelligence, Springer Science & Business, Media, 2006.
32.
WangP., MalaveV. and CipolliniB., Encoding voxels with deep learning, The Journal of Neuroscience35 (2015), 15769–15771. doi:10.1523/JNEUROSCI.3454-15.2015.
33.
ZafarR., MalikA.S., KamelN., DassS.C., AbdullahJ.M., RezaF. and KarimA.H.A., Decoding of visual information from human brain activity: A review of fMRI and EEG studies, Journal of integrative neuroscience (2015), 1–14.
