At the core of Deep Learning-based Deformable Medical Image Registration (DMIR) lies a strong foundation. Essentially, this network compares features in two images to identify their mutual correspondence, which is necessary for precise image registration. In this paper, we use three novel techniques to increase the registration process and enhance the alignment accuracy between medical images. First, we propose cross attention over multi-layers of pairs of images, allowing us to take out the correspondences between them at different levels and improve registration accuracy. Second, we introduce a skip connection with residual blocks between the encoder and decoder, helping information flow and enhancing overall performance. Third, we propose the utilization of cascade attention with residual block skip connections, which enhances information flow and empowers feature representation. Experimental results on the OASIS data set and the LPBA40 data set show the effectiveness and superiority of our proposed mechanism. These novelties contribute to the enhancement of 3D DMIR-based on unsupervised learning with potential implications in clinical practice and research.
KleinS.StaringM.MurphyK.ViergeverM.A.PluimJ.P.W., elastix: A toolbox for intensity-based medical image registration, IEEE Transactions on Medical Imaging29(1) (2010), 196–205. doi: 10.1109/TMI.2009.2035616.
2.
ParraN.A., Rigid and non-rigid point-based medical image registration, PhD thesis, Florida International University, 2009.
3.
MakelaT.ClarysseP.SipilaO.PaunaN.PhamQ.C.KatilaT.MagninI.E., A review of cardiac image registration methods, IEEE Transactions on Medical Imaging21(9) (2002), 1011–1021.
4.
GongM.ChenS.ChenQ.ZengY.ZhangY., Generative adversarial networks in medical image processing, Current Pharmaceutical Design27(15) (2021), 1856–1868.
5.
MahapatraD.AntonyB.SedaiS.GarnaviR., Deformable medical image registration using generative adversarial networks, in: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), IEEE, 2018, pp. 1449–1453.
6.
SinghN.K.RazaK., Medical image generation using generative adversarial networks: A review, Health informatics: A computational perspective in healthcare, 2021, 77–96.
7.
SinghN.K.RazaK., Medical image generation using generative adversarial networks, arXiv preprint arXiv:2005. 10687, 2020.
8.
HaskinsG.KrugerU.YanP., Deep learning in medical image registration: A survey, Machine Vision and Applications31(1–2) (2020). doi: 10.1007/s00138-020-01060-x.
9.
JaderbergM.SimonyanK.ZissermanA. et al., Spatial transformer networks, Advances in Neural Information Processing Systems28 (2015).
10.
BalakrishnanG.ZhaoA.SabuncuM.R.GuttagJ.DalcaA.V., An unsupervised learning model for deformable medical image registration, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9252–9260.
11.
RonnebergerO.FischerP.BroxT., U-net: Convolutional networks for biomedical image segmentation, in: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, Springer, 2015, pp. 234–241.
12.
LiL.HeF.FanR.FanB.YanX., 3D reconstruction based on hierarchical reinforcement learning with transferability, Integrated Computer-Aided Engineering30 (2023), 1–13. doi: 10.3233/ICA-230710.
13.
CruzR.S.LebratL.BourgeatP.FookesC.FrippJ.SalvadoO., DeepCSR: A 3D Deep Learning Approach for Cortical Surface Reconstruction, in: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 806–815. doi: 10.1109/WACV48630.2021.00085.
14.
ChenJ.HeY.FreyE.C.LiY.DuY., Vit-v-net: Vision transformer for unsupervised volumetric medical image registration, arXiv preprint arXiv:2104.06468, 2021.
15.
BalakrishnanG.ZhaoA.SabuncuM.R.GuttagJ.DalcaA.V., VoxelMorph: A learning framework for deformable medical image registration, IEEE Transactions on Medical Imaging38(8) (2019), 1788–1800.
16.
ZhangY.PeiY.ZhaH., Learning dual transformer network for diffeomorphic registration, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, Springer, 2021, pp. 129–138.
17.
ZhouZ.Rahman SiddiqueeM.M.TajbakhshN.LiangJ., Unet: A nested u-net architecture for medical image segmentation, in: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, Springer, 2018, pp. 3–11.
18.
HuangH.LinL.TongR.HuH.ZhangQ.IwamotoY.HanX.ChenY.-W.WuJ., Unet 3: A full-scale connected unet for medical image segmentation, in: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 1055–1059.
19.
ZhaoS.DongY.ChangE.I.XuY. et al., Recursive cascaded networks for unsupervised medical image registration, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 10600–10610.
20.
ZhaoS.LauT.LuoJ.EricI.ChangC.XuY., Unsupervised 3D end-to-end medical image registration with volume tweening network, IEEE Journal of Biomedical and Health Informatics24(5) (2019), 1394–1404.
21.
ThirionJ.-P., Image matching as a diffusion process: an analogy with Maxwell’s demons, Medical Image Analysis2(3) (1998), 243–260.
22.
SotirasA.DavatzikosC.ParagiosN., Deformable medical image registration: A survey, IEEE Transactions on Medical Imaging32(7) (2013), 1153–1190.
ViolaP.Wells IIIW.M., Alignment by maximization of mutual information, International Journal of Computer Vision24(2) (1997), 137–154.
25.
AvantsB.B.EpsteinC.L.GrossmanM.GeeJ.C., Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain, Medical Image Analysis12(1) (2008), 26–41.
26.
LeowA.D.YanovskyI.ChiangM.-C.LeeA.D.KlunderA.D.LuA.BeckerJ.T.DavisS.W.TogaA.W.ThompsonP.M., Statistical properties of Jacobian maps and the realization of unbiased large-deformation nonlinear image registration, IEEE Transactions on Medical Imaging26(6) (2007), 822–832.
27.
KabusS.KlinderT.MurphyK.van GinnekenB.LorenzC.PluimJ.P., Evaluation of 4D-CT lung registration, in: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009: 12th International Conference, London, UK, September 20–24, 2009, Proceedings, Part I 12, Springer, 2009, pp. 747–754.
28.
GreerH.KwittR.VialardF.-X.NiethammerM., ICON: Learning regular maps through inverse consistency, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3396–3405.
29.
ShenZ.VialardF.-X.NiethammerM., Region-specific diffeomorphic metric mapping, Advances in Neural Information Processing Systems32 (2019).
30.
KimY.WisemanS.MillerA.SontagD.RushA., Semi-amortized variational autoencoders, in: International Conference on Machine Learning, PMLR, 2018, pp. 2678–2687.
31.
CremerC.LiX.DuvenaudD., Inference suboptimality in variational autoencoders, in: International Conference on Machine Learning, PMLR, 2018, pp. 1078–1086.
32.
YangX.KwittR.StynerM.NiethammerM., Quicksilver: Fast predictive image registration – a deep learning approach, NeuroImage158 (2017), 378–396.
33.
RohéM.-M.DatarM.HeimannT.SermesantM.PennecX., SVF-Net: learning deformable image registration using shape matching, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11–13, 2017, Proceedings, Part I 20, Springer, 2017, pp. 266–274.
34.
CaoX.YangJ.WangL.XueZ.WangQ.ShenD., Deep learning based inter-modality image registration supervised by intra-modality similarity, in: Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 9, Springer, 2018, pp. 55–63.
35.
KrebsJ.MansiT.MailhéB.AyacheN.DelingetteH., Unsupervised probabilistic deformation modeling for robust diffeomorphic registration, in: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, Springer, 2018, pp. 101–109.
36.
De VosB.D.BerendsenF.F.ViergeverM.A.SokootiH.StaringM.IšgumI., A deep learning framework for unsupervised affine and deformable image registration, Medical Image Analysis52 (2019), 128–143.
37.
LeiY.FuY.WangT.LiuY.PatelP.CurranW.J.LiuT.YangX., 4D-CT deformable image registration using multiscale unsupervised deep learning, Physics in Medicine & Biology65(8) (2020), 085003.
38.
DalcaA.V.BalakrishnanG.GuttagJ.SabuncuM.R., Unsupervised learning for fast probabilistic diffeomorphic registration, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018: 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part I, Springer, 2018, pp. 729–738.
39.
ChenC.-F.R.FanQ.PandaR., Crossvit: Cross-attention multi-scale vision transformer for image classification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 357–366.
40.
XuX.WangT.YangY.ZuoL.ShenF.ShenH.T., Cross-modal attention with semantic consistence for image-text matching, IEEE Transactions on Neural Networks and Learning Systems31(12) (2020), 5412–5425.
41.
LiuY.ZuoL.HanS.XueY.PrinceJ.L.CarassA., Coordinate translator for learning deformable medical image registration, in: International Workshop on Multiscale Multimodal Medical Imaging, Springer, 2022, pp. 98–109.
42.
SongX.ChaoH.XuX.GuoH.XuS.TurkbeyB.WoodB.J.SanfordT.WangG.YanP., Cross-modal attention for multi-modal image registration, Medical Image Analysis82 (2022), 102612.
43.
ShiJ.HeY.KongY.CoatrieuxJ.-L.ShuH.YangG.LiS., Xmorpher: Full transformer for deformable medical image registration via cross attention, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VI, Springer, 2022, pp. 217–226.
44.
DollárP.WelinderP.PeronaP., Cascaded pose regression, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2010, pp. 1078–1085.
45.
FelzenszwalbP.F.GirshickR.B.McAllesterD., Cascade object detection with deformable part models, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Ieee, 2010, pp. 2241–2248.
46.
YuX.ZhouF.ChandrakerM., Deep deformation network for object landmark localization, in: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part V 14, Springer, 2016, pp. 52–70.
47.
CaiZ.VasconcelosN., Cascade r-cnn: Delving into high quality object detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6154–6162.
48.
SchlemperJ.CaballeroJ.HajnalJ.V.PriceA.N.RueckertD., A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Transactions on Medical Imaging37(2) (2017), 491–503.
49.
RavishankarH.VenkataramaniR.ThiruvenkadamS.SudhakarP.VaidyaV., Learning and incorporating shape models for semantic segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2017, pp. 203–211.
50.
ChenH.DouQ.WangX.QinJ.HengP., Mitosis detection in breast cancer histology images via deep cascaded networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30, 2016.
51.
HeK.ZhangX.RenS.SunJ., Deep Residual Learning for Image Recognition, 2015.
52.
MarcusD.S.WangT.H.ParkerJ.CsernanskyJ.G.MorrisJ.C.BucknerR.L., Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults, Journal of Cognitive Neuroscience19(9) (2007), 1498–1507. doi: 10.1162/jocn.2007.19.9.1498.
53.
ShattuckD.W.MirzaM.AdisetiyoV.HojatkashaniC.SalamonG.NarrK.L.PoldrackR.A.BilderR.M.TogaA.W., Construction of a 3D probabilistic atlas of human cortical structures, Neuroimage39(3) (2008), 1064–1080.
54.
KleinS.StaringM.MurphyK.ViergeverM.A.PluimJ.P., Elastix: A toolbox for intensity-based medical image registration, IEEE Transactions on Medical Imaging29(1) (2009), 196–205.
55.
ChenJ.FreyE.C.HeY.SegarsW.P.LiY.DuY., Transmorph: Transformer for unsupervised medical image registration, Medical Image Analysis82 (2022), 102615.
56.
ChenZ.ZhengY.GeeJ.C., TransMatch: A Transformer-based Multilevel Dual-Stream Feature Matching Network for Unsupervised Deformable Image Registration, IEEE Transactions on Medical Imaging, 2023.
NaidenA.PaunescuV.KimG.JeonB.LeordeanuM., Shift R-CNN: Deep Monocular 3D Object Detection With Closed-Form Geometric Constraints, in: 2019 IEEE International Conference on Image Processing (ICIP), 2019, pp. 61–65. doi: 10.1109/ICIP.2019.8803397.
