The rapidly evolving field of photoacoustic tomography utilizes endogenous chromophores to extract both functional and structural information from deep within tissues. It is this power to perform precise quantitative measurements in vivo—with endogenous or exogenous contrast—that makes photoacoustic tomography highly promising for clinical translation in functional brain imaging, early cancer detection, real-time surgical guidance, and the visualization of dynamic drug responses. Considering photoacoustic tomography has benefited from numerous engineering innovations, it is of no surprise that many of photoacoustic tomography’s current cutting-edge developments incorporate advances from the equally novel field of artificial intelligence. More specifically, alongside the growth and prevalence of graphical processing unit capabilities within recent years has emerged an offshoot of artificial intelligence known as deep learning. Rooted in the solid foundation of signal processing, deep learning typically utilizes a method of optimization known as gradient descent to minimize a loss function and update model parameters. There are already a number of innovative efforts in photoacoustic tomography utilizing deep learning techniques for a variety of purposes, including resolution enhancement, reconstruction artifact removal, undersampling correction, and improved quantification. Most of these efforts have proven to be highly promising in addressing long-standing technical obstacles where traditional solutions either completely fail or make only incremental progress. This concise review focuses on the history of applied artificial intelligence in photoacoustic tomography, presents recent advances at this multifaceted intersection of fields, and outlines the most exciting advances that will likely propagate into promising future innovations.
WangLVYaoJ.A practical guide to photoacoustic tomography in the life sciences. Nat Methods2016;
13:627–38
6.
BellAG.On the production and reproduction of sound by light. Am J Sci1880;
s3-20:305–24
7.
ManoharSRDanielPB.Photoacoustics: a historical review. Adv Opt Photon2016;
8:586–617
8.
XuMWangLV.Photoacoustic imaging in biomedicine. Rev Sci Instrum2006;
77:041101
9.
WangLV.Multiscale photoacoustic microscopy and computed tomography. Nat Photon2009;
3:503–9
10.
DuarteMFDavenportMATakharDLaskaJNSunTKellyKFBaraniukRG.Single-pixel imaging via compressive sampling. IEEE Signal Process Mag2008;
25:83–91
11.
LiangJZhouYWinklerAWWangLMaslovKILiCWangLV.Random-access optical-resolution photoacoustic microscopy using a digital micromirror device. Opt Lett2013;
38:2683–6
12.
HaltmeierMBererTMoonSBurgholzerP.Compressed sensing and sparsity in photoacoustic tomography. J Opt2016;
18:114004
13.
JeonSKimJLeeDBaikJWKimC.Review on practical photoacoustic microscopy. Photoacoustics2019;
15:100141
14.
KimJKimJYJeonSBaikJWChoSHKimC.Super-resolution localization photoacoustic microscopy using intrinsic red blood cells as contrast absorbers. Light Sci Appl2019;
8:103
15.
AgranovskyMKuchmentP.Uniqueness of reconstruction and an inversion procedure for thermoacoustic and photoacoustic tomography. Inverse Prob2007;
23:2089–102
16.
AntholzerSHaltmeierMSchwabJ.Deep learning for photoacoustic tomography from sparse data. Inverse Probl Sci Eng2019;
27:987–1005
HauptmannALuckaFBetckeMHuynhNAdlerJCoxBBeardPOurselinSArridgeS.Model-based learning for accelerated, limited-view 3-d photoacoustic tomography. IEEE Trans Med Imaging2018;
37:1382–93
DiSpiritoAIIILiDVuTChenMZhangDLuoJHorstmeyerRYaoJ.Reconstructing undersampled photoacoustic microscopy images using deep learning. IEEE Trans Med Imaging2020;
40:562–70
21.
ChenXQiWXiL.Deep-learning-based motion-correction algorithm in optical resolution photoacoustic microscopy. Vis Comput Ind Biomed Art2019;
2:12
22.
SharmaAPramanikM.Convolutional neural network for resolution enhancement and noise reduction in acoustic resolution photoacoustic microscopy. Biomed Opt Express2020;
11:6826–39
23.
GoodfellowIBengioYCourvilleA.Deep learning.
Cambridge, MA:
MIT Press, 2016
24.
SchmidhuberJ.Deep learning in neural networks: an overview. Neural Netw2015;
61:85–117
RazzakMINazSZaibA.Deep learning for medical image processing: overview, challenges and the future. In: DeyNAshourASBorraS (eds) Classification in BioApps: automation of decision making.
Cham:
Springer International Publishing, 2018, pp. 323–50
27.
LitjensGKooiTBejnordiBESetioAAACiompiFGhafoorianMvan der LaakJvan GinnekenBSanchezCI.A survey on deep learning in medical image analysis. Med Image Anal2017;
42:60–88
28.
NajafabadiMMVillanustreFKhoshgoftaarTMSeliyaNWaldRMuharemagicE.Deep learning applications and challenges in big data analytics. J Big Data2015;
2:1--21
29.
LeeJ-GJunSChoYWLeeHKimGBSeoJBKimN.Deep learning in medical imaging: general overview. Korean J Radiol2017;
18:570
EricksonBJKorfiatisPAkkusZKlineTL.Machine learning for medical imaging. RadioGraphics2017;
37:505–15
32.
SahinerBPezeshkAHadjiiskiLMWangXDrukkerKChaKHSummersRMGigerML.Deep learning in medical imaging and radiation therapy. Med Phys2019;
46:e1–e36
33.
Martorell-MarugánJSihamTYassirBCoral delVIgorZFranciscoHPedroC-SHolgerH.Deep learning in omics data analysis and precision medicine. Comput Biol2019;
3:37–53
34.
ShenCNguyenDZhouZJiangSBDongBJiaX.An introduction to deep learning in medical physics: advantages, potential, and challenges. Phys Med Biol2020;
65:05TR01
35.
PradhanPGuoSRyabchykovOPoppJBocklitzTW.Deep learning a boon for biophotonics?J Biophotonics2020;
13:e201960186
WaibelDGröhlJIsenseeFKirchnerTMaier-HeinKMaier-HeinL. Reconstruction of initial pressure from limited view photoacoustic images using deep learning. In: Oraevsky AA, Wang LV (eds) Photons Plus Ultrasound: Imaging and Sensing 2018. International Society for Optics and Photonics, SPIE, vol. 10494, 2018, p.104942S
38.
AnasEMAZhangHKAudigierCBoctorEM.Robust photoacoustic beamforming using dense convolutional neural networks.
Cham:
Springer International Publishing, 2018, pp. 3–11
39.
FengJDengJLiZSunZDouHJiaK.End-to-end Res-Unet based reconstruction algorithm for photoacoustic imaging. Biomed Opt Express2020;
11:5321–40
40.
GuanSKhanAASikdarSChitnisPV.Limited-View and sparse photoacoustic tomography for neuroimaging with deep learning. Sci Rep2020;
10:1–12
41.
KimMWJengG-SPelivanovIO’DonnellM.Deep-learning image reconstruction for real-time photoacoustic system. IEEE Trans Med Imag2020;
39:3379–90
42.
LanHJiangDYangCGaoFGaoF.Y-Net: hybrid deep learning image reconstruction for photoacoustic tomography in vivo. Photoacoustics2020;
20:100197
43.
WangZBovikACSheikhHRSimoncelliEP.Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process2004;
13:600–12
44.
AwasthiNJainGKalvaSKPramanikMYalavarthyPK.Deep neural network based sinogram super-resolution and bandwidth enhancement for limited-data photoacoustic tomography. IEEE Trans Ultrason Ferroelectr Freq Control2020;
67:2660–73
45.
ReiterABellMAL.A machine learning approach to identifying point source locations in photoacoustic data. Photons Plus Ultrasound: Imag Sens2017;
10064:100643J
46.
AllmanDReiterABellMAL.Photoacoustic source detection and reflection artifact removal enabled by deep learning. IEEE Trans Med Imaging2018;
37:1464–77
47.
AllmanDAssisFChrispinJBellMAL.A deep learning-based approach to identify in vivo catheter tips during photoacoustic-guided cardiac interventions. Photons Plus Ultrasound: Imag Sens2019;
10878: 108785E
48.
GuanSKhanAASikdarSChitnisPV.Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform2019;
24:568–76
49.
LanHZhouKYangCChengJLiuJGaoSGaoF. Ki-GAN: knowledge infusion generative adversarial network for photoacoustic image reconstruction in vivo. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 13--17 October 2019, pp.273–81. Berlin: Springer
50.
SchwabJAntholzerSNusterRPaltaufGHaltmeierM.Deep Learning of truncated singular values for limited view photoacoustic tomography. Photons Plus Ultrasound: Imag Sens2019;
10878:1087836
51.
AwasthiNPrabhakarKRKalvaSKPramanikMBabuRVYalavarthyPK.PA-Fuse: deep supervised approach for the fusion of photoacoustic images with distinct reconstruction characteristics. Biomed Opt Express2019;
10:2227–43
52.
AntholzerSSchwabJBauer-MarschallingerJBurgholzerPHaltmeierM.NETT regularization for compressed sensing photoacoustic tomography. SPIE BiOS2019;
10878:108783B
53.
VuTLiMHumayunHZhouYYaoJ.A generative adversarial network for artifact removal in photoacoustic computed tomography with a linear-array transducer. Exp Biol Med2020;
245:597–605
54.
GodefroyGArnalBBossyE.Compensating for visibility artefacts in photoacoustic imaging with a deep learning approach providing prediction uncertainties. Photoacoustics2020;
21:100218
55.
ZhangHHongyuLNyayapathiNWangDLeAYingLXiaJ.A new deep learning network for mitigating limited-view and under-sampling artifacts in ring-shaped photoacoustic tomography. Comput Med Imaging Graph2020;
84:101720
56.
LuTChenTGaoFSunBNtziachristosVLiJL.LV‐GAN. A deep learning approach for limited‐view optoacoustic imaging based on hybrid datasets. J Biophotonics2020;
14:e202000325
57.
SinghMKASivasubramanianKSatoNIchihashiFSankaiYXingL.Deep learning-enhanced LED-based photoacoustic imaging.Photons Plus Ultrasound: Imag Sens2020;
11240:1124038
58.
AnasEMAZhangHKKangJBoctorE.Enabling fast and high quality LED photoacoustic imaging: a recurrent neural networks based approach. Biomed Opt Express2018;
9:3852–66
RajendranPPramanikM.Deep learning approach to improve tangential resolution in photoacoustic tomography. Biomed Opt Express2020;
11:7311–23
62.
TongTHuangWWangKHeZYinLYangXZhangSTianJ.Domain transform network for photoacoustic tomography from limited-view and sparsely sampled data. Photoacoustics2020;
19:100190
63.
BoinkYEManoharSBruneC.A Partially-Learned algorithm for joint photo-acoustic reconstruction and segmentation. IEEE Trans Med Imaging2019;
39:129–39
64.
HauptmannACoxBLuckaFHuynhNBetckeMBeardPArridgeS.Approximate k-Space models and deep learning for fast photoacoustic reconstruction.
Cham:
Springer International Publishing, 2018, pp. 103–11
65.
YangCLanHGaoF. Accelerated photoacoustic tomography reconstruction via recurrent inference machines. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23--27 July 2019, pp.6371–74. Piscataway, NJ: IEEE.
66.
YangCGaoF. Eda-net: dense aggregation of deep and shallow information achieves quantitative photoacoustic blood oxygenation imaging deep in human breast. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A (eds) International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2019, pp. 246–54
67.
CaiCDengKMaCLuoJ.End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging. Opt Lett2018;
43:2752–5
68.
YangCLanHZhongHGaoF. Quantitative photoacoustic blood oxygenation imaging using deep residual and recurrent neural network. In: 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, 8--11 April 2019, pp.741–44. Piscataway, NJ: IEEE
69.
OlefirITzoumasSRestivoCMohajeraniPXingLNtziachristosV.Deep learning-based spectral unmixing for optoacoustic imaging of tissue oxygen saturation. IEEE Trans Med Imaging2020;
39:3643–54
70.
BenchCHauptmannACoxBT.Toward accurate quantitative photoacoustic imaging: learning vascular blood oxygen saturation in three dimensions. J Biomed Opt2020;
25:085003
71.
GovinahallisathyanarayanaSNingBCaoRHuSHossackJA.Dictionary learning-based reverberation removal enables depth-resolved photoacoustic microscopy of cortical microvasculature in the mouse brain. Sci Rep2018;
8:985
