Sage Journals: Discover world-class research

Abstract

There is a lot of Chinese information on the Internet, including many architectural texts. In the field of building information, how to effectively find and organize these architectural texts has become a hot research topic. In this paper, the parallel optimizing of Deep Belief Network (DBN) implemented by Spark is proposed to classify the architectural texts. Firstly, the architectural texts preprocessing is automatically parallelized by spark technology. Secondly, the Word2Vec tool is used to represent the architectural texts and construct the feature dictionary. Thirdly the architectural text classification model is constructed by the Deep Belief Network. Finally, architectural text classification is parallelized by the Spark cluster. From experimental results we can see that the training time of parallel DBN is shortened to a quarter of the original, and gets less errors and better performance than that of the traditional shallow learning method.

Keywords

Spark parallelization Deep Belief Network architectural texts classification

Get full access to this article

View all access options for this article.

References

Engle

Lupher

Xin

et al., Shark: fast data analysis using coarse-grained distributed memory [C], in: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, ACM, 2012, pp. 689–692.

Zhang

C.S.

, Challenges in Machine Learning [J], Science China (Information Sciences) 12 (2013), 1612–1623.

Zhang

C.X.

N.N.

and Wang

G.W.

, Introduction of Restricted Boltzmann Machine [C], Chinese Science and Technology Papers Online, 2010.

Fang

Cai

Z.P.

Zhao

Q.J.

Lin

J.R.

and Zhu

, Adaptive Technique for Real-Time DDoS Detection and Defense Using Spark Streaming [J], Journal of Frontiers of Computer Science and Technology 10(5) (2016), 601–611.

Hinton

G.E.

, Training products of experts by minimizing contrastive divergence [J], Neural Computation 14(7) (2002), 1771–1800.

Hinton

G.E.

and Salakhutdinov

R.R.

, Reducing the Dimensionality of Data with Neural Networks [J], Science 28(6) (2006), 504–507.

Hinton

G.E.

and Salakhutdinov

R.R.

, Reducing the dimensionality of data with neural networks, Science 313(5786) (2006), 504–507.

Hinton

G.E.

and Osindero

, A fast learning algorithm for deep belief nets [J], Neural Computation 18(6) (2006), 1527–1554.

Lee

Grosse

Ranganath

et al., Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, in: Proc. of the 26th Annual International Conference on Machine Learning, ACM, 2009, pp. 609–616.

10.

Ding

Liu

Chen

Feng

and Wang

Y.H.

, Similarity Constrained Deep Belief Networks with Application to SAR Image Target Recognition [J], Journal of Electronics & Information Technology 38(1) (2016), 97–102.

11.

Jia

Chen

Y.Q.

and Xu

, Deep Learning: Yesterday, Today, and Tomorrow [J], Journal of Computer Research and Development 50(8) (2013), 1799–1804.

12.

Latest Release Spark Documentation [EB/OL]. http://spark.apache.org/docs/latest/, 2015.

13.

Abdel-Rahman

George

and Geoffrey

, Acoustic modeling using deep belief networks [J], IEEE Transactions on Audio, Speech, and Language Processing 20(1) (2012), 14–22.

14.

P.P.

J.C.

Wei

J.T.

Cao

and Yao

B.X.

, KNN Text Classification Algorithm Based on Spark and Word Relevancy [J], Computer Technology and Development 28(3) (2018), 87–92.

15.

Zhu

Y.H.

and Lu

, SVM-based Sentinel Sentence Recognition Algorithm [J], Journal of Hunan University of Technology 27(2) (2013), 89–93.

16.

Socher

Perelygin

J.Y.

and Chuang

, Recursive deep models for semantic compositionality over a sentiment treebank, in: Proc. of the Conf. on Empirical Methods in Natural Language Processing (EMNLP), 2013, pp. 1631–1642.

17.

Socher

Pennington

Eric

H.H.

et al., Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions [C], in: Proc. of the Conf. on Empirical Methods in Natural Language Processing (EMNLP), 2011, pp. 151–161.

18.

Socher

Pennington

Eric

H.H.

et al., Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions [C], EMNLP, 2011.

19.

Mikolov

Chen

Corrado

et al., Efficient estimation of word representations in vector space//proceedings of workshop at ICLR, 2013.

20.

Huang

W.H.

and Feng

, A Novel Performance Evaluation Method for Processing Video/Image Stream Based on Spark Streaming [J], Computer Engineer & Scienc 37(11) (2016), 2055–2060.

21.

Chen

Zheng

D.Q.

and Zhao

T.J.

, Chinese Relation Extraction Based on Deep Belief Nets [J], Journal of Software 2012(10), 2572–2585.

22.

Y.X.

, Design and Implement Recommend System Based on Spark [D], Zhe Jiang University, 2015.

Application of Spark parallelization technology in architectural text classification

Abstract

Keywords

Get full access to this article

References