This paper intends to perform de-duplication for enhancing the storage optimization. Hence, this paper contributes by proposing a hybrid fingerprint extracting using simhash (SH) and Huffman coding (HC) algorithms. Secondly, the data is clustered using the latest technique called as grey wolf optimization (GWO) to extract the metadata. The extracted metadata is stored in metadata server which provides better storage optimization and de-duplication. Euclidean distance based GWO is adopted as it provides minimum Euclidean distance in the GWO based clustering for de-duplication. The proposed GWO based clustering method is compared with the existing methods such as k-means, k-mode, Euclidean distance based Particle Swarm Optimization and Euclidean distance based genetic algorithm in terms of accuracy, True Positive Rate (TPR), True Negative Rate (TNR) and performance time and the significance of the GWO based clustering method is described.
XingY.-X.XiaoN.LiuF.SunZ. and HeW.-H., AR-dedupe: An efficient de-duplication approach for cluster de-duplication system, Journal of Shanghai Jiaotong University (Science)20(1) (2015), 76–81.
2.
ZhangY. et al., A fast asymmetric extremum content defined chunking algorithm for data de-duplication in backup storage systems, IEEE Transactions on Computers66(2) (2017), 199–211.
3.
WangJ.ZhaoZ.XuZ.ZhangH.LiL. and GuoY., I-sieve: An inline high performance de-duplication system used in cloud storage, in Tsinghua Science and Technology20(1) (2015), 17–27.
4.
ZhangP.HuangP.HeX.WangH. and ZhouK., Resemblance and mergence based indexing for high performance data de-duplication, Journal of Systems and Software128 (2017), 11–24.
5.
HeK.HuangC.ZhouH.ShiJ.WangX. and DanF., Public auditing for encrypted data with client-side de-duplication in cloud storage, Wuhan University Journal of Natural Sciences20(4) (2015), 291–298.
6.
KwonH.HahnC.KimD. and HurJ., Secure de-duplication for multimedia data with user revocation in cloud storage, Multimedia Tools and Applications76(4) (2017), 5889–5903.
7.
LiX.LiJ. and HuangF., A secure cloud storage system supporting privacy-preserving fuzzy de-duplication, Soft Computing20(4) (2016), 1437–1448.
8.
QuinlanS. and DorwardS., Venti: A new approach to archival storage, Proceedings of the FAST ’02 Conference on File and Storage Technologies, January 28-30, 2002, Doubletree Hotel, Monterey, California, USA. USENIX FAST.
9.
TolicA. and BrodnikA., De-duplication in unstructured-data storage systems, Elektrotehniski Vestnik82(5) (2015), 233–242.
10.
BellareM.KeelveedhiS. and RistenpartT., Message-locked encryption and secure de-duplication, Advances in Cryptology – EUROCRYPT7881 (2013), 296–312.
11.
XiaW. et al., A comprehensive study of the past, present, and future of data de-duplication, Proceedings of the IEEE104(9) (2016), 1681–1710.
12.
EshghiK. and TangH.K., A framework for analyzing and improving content-based chunking algorithms, Hewlett-Packard Labs Technical Report TR30 (2005).
13.
XiaW.JiangH.FengD. and TianL., Combining de-duplication and delta compression to achieve low-overhead data reduction on backup datasets, 2014 Data Compression Conference, Snowbird, UT (2014), 203–212.
14.
XiaW.JiangH.FengD.TianL.FuM. and WangZ., P-De-dupe: Exploiting parallelism in data de-duplication system, 2012 IEEE Seventh International Conference on Networking, Architecture, and Storage, Xiamen, Fujian (2012), 338–347.
15.
BjørnerN.BlassA. and GurevichY., Content-dependent chunking for differential compression, the local maximum approach, Journal of Computer and System Sciences76(3) (2010), 154–203.
16.
AnandA.MuthukrishnanC.AkellaA. and RamjeeR., Redundancy in network traffic: Findings and implications, in Proc ACM SIGMETRICS, Seattle, WA, USA (2009).
17.
LuG.JinY. and DuD.H.C., Frequency based chunking for data de-duplication, 2010 IEEE International Symposium on Modelling, Analysis and Simulation of Computer and Tele communication Systems, Miami Beach, FL (2010), 287–296.
18.
ZhouB. and WenJ., Hysteresis re-chunking based metadata harnessing de-duplication of disk images, 2013 42nd International Conference on Parallel Processing, Lyon (2013), 389–398.
19.
BobbarjungD.R.JagannathanS. and DubnickiC., Improving duplicate elimination in storage systems, ACM Transactions on Storage (TOS)2(4) (2006), 424–448.
LiuC.XueY.JuD. and WangD., A novel optimization method to improve de-duplication storage system performance, 2009 15th International Conference on Parallel and Distributed Systems, Shenzhen (2009), 228–235.
22.
LuG.NamY.J. and DuD.H.C., BloomStore: Bloom-Filter based memory-efficient key-value store for indexing of data de-duplication on flash, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST), San Diego, CA (2012), 1–11.
23.
LiuC.LuY.ShiC.LuG.DuD.H.C. and WangD.S., ADMAD: Application-driven metadata aware de-duplication archival storage system, 2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os, Baltimore, MD (2008), 29–35.
24.
FuY.JiangH.XiaoN.TianL.LiuF. and XuL., Application-aware local-global source de-duplication for cloud backup services of personal storage, IEEE Transactions on Parallel and Distributed Systems25(5) (2014), 1155–1165.
25.
LiJ.LiY.K.ChenX.LeeP.P.C. and LouW., A hybrid cloud approach for secure authorized de-duplication, IEEE Transactions on Parallel and Distributed Systems26(5) (2015), 1206–1216.
26.
KaczmarczykM. and DubnickiC., Reducing fragmentation impact with forward knowledge in backup systems with de-duplication, Proceedings of the 8th International Systems and Storage Conference (SYSTOR’15), IEEE, Haifa, Israel (2015), 201–208.
27.
MirjaliliS.MirjaliliS.M. and LewisA., Grey Wolf optimizer, Advances in Engineering Software69 (2014), 46–61.
28.
kumarJ.P. and GovindharajuluP., Near-duplicate web page detection: An efficient approach using clustering, sentence feature and fingerprinting, International Journal of Computational Intelligence Systems6(1) (2013), 1–13.
29.
KudovaP., Clustering genetic algorithm, 18th International Workshop on Database and Expert Systems Applications, IEEE Conference Publications, Regensburg, Germany (2007), 138–142.
30.
GanG. and Kwok-Po NgM., k-means clustering with outlier removal, Pattern Recognition Letters90 (2017), 8–14.
31.
JiangF.LiuG.DuJ. and SuiY., Initialization of k-modes clustering using outlier detection techniques, Original Research Article Information Sciences332 (2016), 167–183.
32.
MankuG.S.JainA. and SarmaA.D., Detecting near duplicatesfor web crawling, Proceedings of the 16th international conference on World Wide Web, Banff, Alberta, Canada (2007), 141–150.
33.
Huffman coding and Huffman tree, www.csc.lsu.edu/∼kundu/dstr/4-huffman.pdf.
34.
ParkD.FanZ.NamY.J. and DuD.H.C., A lookahead read cache: Improving read performance for deduplication backup storage, Journal of Computer Science and Technology32(1) (2017), 26–40.
35.
WuF.FanZ.YangM.C.ZhangB.GeX. and DuD.H.C., Performance evaluation of host aware shingled magnetic recording (HA-SMR) drives, IEEE Transactions on Computers66(11) (2017), 1932–1945.
36.
FanZ.WuF.ParkD.DiehlJ.VoigtD. and DuD.H., Hibachi: A cooperative hybrid cache with nvram and dram for storage arrays, In Proc of IEEE Conference on Mass Storage Systems and Technologies (MSST) (2017).
37.
KumarB.S.S.ManjunathA.S. and ChristopherS., Improved entropy encoding for high efficient video coding standard, Alexandria Engineering Journal57(1) (2018), 1–9.
38.
KotaP.N. and GaikwadA.N., Optimized scrambling sequence to reduce papr in space frequency block codes based MIMO-OFDM system, Journal of Advanced Research in Dynamical and Control System (2017), 502–525.
39.
BhatnagarK. and GuptaS.C., Extending the neural model to study the impact of effective area of optical fiber on laser intensity, International Journal of Intelligent Engineering and Systems10(4) (2017), 274–283.
40.
BalajiG.N.SubashiniT.S. and ChidambaramN., Detection of heart muscle damage from automated analysis of echocardiogram video, IETE Journal of Research61(3), 236–243.
41.
BramheS.S.DalalA.TajneD. and MarotkarD., Glass shaped antenna with defected ground structure for cognitive radio application, International Conference on Computing Communication Control and Automation, Pune (2015), 330–333.
42.
YarrapragadaK.S.S.R. and KrishnaB.B., Impact of tamanu oil-diesel blend on combustion, performance and emissions of diesel engine and its prediction methodology, Journal of the Brazilian Society of Mechanical Sciences and Engineering, 1–15.
43.
SreedharanN.P.N.GanesanB.RaveendranR.SaralaP. and DennisB., Grey Wolf optimisation-based feature selection and classification for facial emotion recognition, IET Biometrics (2018).
44.
SarkarA. and MuruganT.S., Cluster head selection for energy efficient and delay-less routing in wireless sensor network, Wireless Networks (2017), 1–18.
45.
WaghA.M. and TodmalS.R., Eyelids, eyelashes detection algorithm and hough transform method for noise removal in iris recognition, International Journal of Computer Applications112(3) (2015).
46.
IyapparajaM. and TiwariM., Security policy speculation of user uploaded images on content sharing sites, IOP Conference Series: Materials Science and Engineering263(4) (2017), 042019.
