Multi-modal medical image fusion plays a crucial role in many areas of modern medicine like diagnosis and therapy planning.
OBJECTIVE:
Due to the factor that the structure tensor has the property of preserving the image geometry, we utilized it to construct the directional structure tensor and further proposed an improved 3-D medical image fusion method.
METHOD:
The local entropy metrics were used to construct the gradient weights of different source images, and the eigenvectors of traditional structure tensor were combined with the second-order derivatives of image to construct the directional structure tensor. In addition, the guided filtering was employed to obtain detail components of the source images and construct a fused gradient field with the enhanced detail. Finally, the fusion image was generated by solving the functional minimization problem.
RESULTS AND CONCLUSION:
Experimental results demonstrated that this new method is superior to the traditional structure tensor and multi-scale analysis in both visual effect and quantitative assessment.
SchoderH, YeungH.W, GonenM, et al., Head and neck cancer: clinical usefulness and accuracy of PET/CT image fusion, Radiology231 (2004), 65–72.
2.
CaiJ, ChuJ.C, RecineD, et al., CT and PET lung image registration and fusion in radiotherapy treatment planning using the chamfer-matching method, International Journal of Radiation Oncology* Biology* Physics43 (1999), 883–891.
3.
ShahdoostiH.R, MehrabiA, MRI and PET image fusion using structure tensor and dual ripplet-II transform, Multimed Tools Appl77 (2018), 22649–22670.
4.
DonatiO.F, HanyT.F, ReinerC.S, et al., Value of retrospective fusion of PET and MR images in detection of hepatic metastases: comparison with 18F-FDG PET/CT and Gd-EOB-DTPA– enhanced MRI, J Nucl Med51 (2010), 692–699.
5.
GaemperliO, SchepisT, ValentaI, et al., Cardiac image fusion from stand-alone SPECT and CT: clinical experience, J Nucl Med48 (2007), 696–703.
6.
OgataY, NakaharaT, OdeK, et al., 3D SPECT/CT fusion using image data projection of bone SPECT onto 3D volume-rendered CT images: feasibility and clinical impact in the diagnosis of bone metastasis, Ann Nucl Med31 (2017), 304–314.
7.
GrosuA.L, WeberW.A, FranzM, et al., Reirradiation of recurrent high-grade gliomas using amino acid PET (SPECT)/CT/MRI image fusion to determine gross tumor volume for stereotactic fractionated radiotherapy, Int J Radiat Oncol63 (2005), 511–519.
8.
NemecS.F, DonatM.A, MehrainS, et al., CT– MR image data fusion for computer assisted navigated neurosurgery of temporal bone tumors, Eur J Radiol62 (2007), 192–198.
9.
MaX, HuS, LiuS, Multi-focus image fusion based on joint sparse representation and optimum theory, Signal Process-Image78 (2019), 125–134.
10.
LiH, HeX, TaoD, et al., Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning, Pattern Recogn79 (2018), 130–146.
11.
LiS, KangX, HuJ, Image fusion with guided filtering, IEEE T Image Pr22 (2013), 2864–2875.
12.
PiellaG, Image fusion for enhanced visualization: A variational approach, Int J Comput Vi83 (2009), 1–11.
13.
SinghR, KhareA, Fusion of multimodal medical images using Daubechies complex wavelet transform– A multiresolution approach, Inform Fusion19 (2014), 49–60.
14.
ZhuZ.Q, ChaiY, YinH.P, LiA.X, A novel dictionary learning approach for multi-modality medical image fusion, Neurocomputing214 (2016), 471–482.
15.
BhavanaV, KrishnappaH.K, Multi-modality medical image fusion using discrete wavelet transform, Procedia Comput Sci70 (2015), 625–631.
16.
LimW.Q, The discrete shearlet transform: a new directional transform and compactly supported shearlet frames, IEEE T Image Pr19 (2010), 1166–1180.
17.
DoM.N, VetterliM, The contourlet transform: an efficient directional mul-tiresolution image representation, IEEE T Image Pr14 (2005), 2091–2106.
18.
Da CunhaA.L, ZhouJ, DoM.N, The non-subsampled contourlet transform: theory, design and applications, IEEE T Image Pr15 (2006), 3089–3101.
19.
LiuS, WangJ, LuY, et al., Multi-focus Image Fusion Based on Adaptive Dual-channel Spiking Cortical Model in Non-subsampled Shearlet Domain, IEEE Access7 (2019), 56367–56388.
20.
MaX, HuS, LiuS, et al., Noisy remote sensing image fusion based on JSR, IEEE Access8 (2020), 31069–31082.
21.
LiuS, WangJ, LuY, et al., Multi-Focus Image Fusion Based on Residual Network in Non-Subsampled Shearlet Domain, IEEE Access7 (2019), 152043–152063.
22.
LiuY, ChenX, ChengJ, PengH, A medical image fusion method based on convolutional neural networks, IEEE 20th International Conference on Information Fusion (2017), 1–7.
23.
HuangP.W, ChenC.I, ChenP, et al., PET and MRI brain image fusion using wavelet transform with structural information adjustment and spectral information patching, IEEE International Symposium on Bioelectronics and Bioinformatics (IEEE ISBB 2014) IEEE, 2014, 1–4.
WangL, LiB, TianL, Multimodal Medical Volumetric Data Fusion Using 3-D Discrete Shearlet Transform and Global-to-Local Rule, IEEE T Bio-Med Eng61 (2013), 197–206.
26.
WangY.J, LiuY, Three-dimensional structure tensor based PET/CT fusion in gradient domain, J X-ray Sci Tec27 (2019), 307–319.
27.
YinH, Tensor sparse representation for 3-D medical image fusion using weighted average rule, IEEE T Bio-Med Eng65 (2018), 2622–2633.
28.
ZhaoW, XuZ, ZhaoJ, Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion, Inform Fusion27 (2016), 138–149.
29.
WangY, WangY.J, Anisotropic diffusion filtering method with weighted directional structure tensor, Biomed Signal P53 (2019), 101590.
30.
WuX, JansonX, Directional structure tensors in estimating seismic structural and stratigraphic orientations, Geophys J Int210 (2017), 534–548.
31.
HeC, WangY, ChenQ, Active contours driven by weighted region-scalable fitting energy based on local entropy, Signal Process92 (2012), 587–600.
32.
ClarkK, VendtB, SmithK, et al., The cancer imaging archive (TCIA): maintaining and operating a public information repository, J Digit Imag26 (2013), 1045–1057.
33.
MenzeB.H, JakabA, BauerS, et al., The multimodal brain tumor image segmentation Benchmark (BRATS), IEEE T Med Imag34 (2015), 1993–2024.
34.
BakasS, AkbariH, SotirasA, et al., Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features, Nature Scientific Data4 (2017), 170117.
35.
LiuX, MeiW, DuH, Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion, Neurocomputing235 (2017), 131–139.
36.
MaJ, ChenC, LiC, HuangJ, Infrared and visible image fusion via gradient transfer and total variation minimization, Inform. Fusion31 (2016), 100–109.
37.
ZhuZ, ZhengM, QiG, et al., A Phase Congruency and Local Laplacian Energy Based Multi-Modality Medical Image Fusion Method in NSCT Domain, IEEE Access7 (2019), 20811–20824.
38.
WangD, LiZ, CaoL, et al., Image fusion incorporating parameter estimation optimized Gaussian mixture model and fuzzy weighted evaluation system: A case study in time-series plantar pressure data set, IEEE Sens J17 (2016), 1407–1420.
