A reliable, punctual, and spatially accurate dataset of sidewalks is vital for identifying where improvements can be made upon urban environment to enhance multi-modal accessibility, social cohesion, and residents' physical activity. This paper develops a synthetically new spatial procedure to extract the sidewalk by integrating the detected results from aerial and street view imagery. We first train neural networks to extract sidewalks from aerial images, and then use pre-trained models to restore occluded and missing sidewalks from street view images. By combining the results from both data sources, a complete network of sidewalks can be produced. Our case study includes four counties in the U.S., and both precision and recall reach about 0.9. The street view imagery helps restore the occluded sidewalks and largely enhances the sidewalk network's connectivity by linking 20% of dangles.
Aceves-GonzálezCEkambaramKRey-GalindoJ, et al. (2020) The role of perceived pedestrian safety on designing safer built environments. Traffic Injury Prevention1–6. https://doi.org/10.1080/15389588.2020.1812062
2.
AwadM (2010) An unsupervised artificial neural network method for satellite image segmentation. Int Arab J Inf Technol7: 199–205.
3.
BadrinarayananVKendallACipollaR (2017) SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence39: 2481–2495.
4.
BallJEAndersonDTChanCS (2017) Comprehensive survey of deep learning in remote sensing: Theories, tools, and challenges for the community. Journal of Applied Remote Sensing11: 042609.
5.
BolyaDZhouCXiaoF, et al. (2019) YOLACT: Real-time instance segmentation. ArXiv190402689 Cs.
6.
BoonpookWTanYYeY, et al. (2018) A deep learning approach on building detection from unmanned aerial vehicle-based images in riverbank monitoring. Sensors18: 3921.
7.
BouzianiMGoïtaKHeD-C (2010) Automatic change detection of buildings in urban environment from very high spatial resolution images using existing geodatabase and prior knowledge. ISPRS Journal of Photogrammetry and Remote Sensing65: 143–153.
8.
BransonSWegnerJDHallD, et al. (2018) From Google Maps to a fine-grained catalog of street trees. ISPRS Journal of Photogrammetry and Remote Sensing135: 13–30.
9.
CaoRZhuJTuW, et al. (2018) Integrating aerial and street view images for urban land use classification. Remote Sensing 10(10). https://doi.org/10.3390/rs10101553
DeyVZhangYZhongM (2010) A Review on Image Segmentation Techniques with Remote Sensing Perspective. In: Proceedings of ISPRS TC VII Symposium – 100 Years ISPRS, Vienna, Austria: 31–42.
16.
ForsythASouthworthM (2008) Cities afoot – Pedestrians, walkability and urban design. Journal of Urban Design13: 1–3.
17.
GaetanoRMasiGPoggiG, et al. (2014) Marker-controlled watershed-based segmentation of multiresolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing53: 2987–3004.
18.
GaoL, SongW, DaiJ, et al. (2019) Road Extraction from High-Resolution Remote Sensing Imagery Using Refined Deep Residual Convolutional Neural Network. Remote Sensing11(5): https://doi.org/10.3390/rs11050552
19.
GhoshSDasNDasI, et al. (2019) Understanding deep learning techniques for image segmentation. ACM Computing Surveys52: 73–73.
20.
HanSRenFWuC, et al. (2018) Using the TensorFlow deep neural network to classify Mainland China visitor behaviors in Hong Kong from check-in data. ISPRS International Journal of Geo-Information 7(4). https://doi.org/10.3390/ijgi7040158
21.
HeKGkioxariGDollárP, et al. (2017) Mask R-CNN. ArXiv170306870 Cs.
22.
HoffmannEJWangYWernerM, et al. (2019) Model fusion for building type classification from aerial and street view images. Remote Sensing11: 1259.
23.
HouQAiC (2020) A network-level sidewalk inventory method using mobile LiDAR and deep learning. Transportation Research Part C: Emerging Technologies119: 102772.
24.
JenningsDFigliozziM (2019) Study of sidewalk autonomous delivery robots and their potential impacts on freight efficiency and travel. Transportation Research Record: Journal of the Transportation Research Board2673: 317–326.
25.
KimSChoiJKimY (2011) Determining the sidewalk pavement width by using pedestrian discomfort levels and movement characteristics. KSCE Journal of Civil Engineering15: 883.
26.
KussulNLavreniukMSkakunS, et al. (2017) Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters14: 778–782.
27.
LiYGuoLRaoJ, et al. (2018) Road segmentation based on hybrid convolutional network for high-resolution visible remote sensing image. IEEE Geoscience and Remote Sensing Letters16: 613–617.
28.
LiYQiHDaiJ, et al. (2017) Fully convolutional instance-aware semantic segmentation. In: Proceedings of 2017 IEEE conference on computer vision and pattern recognition, Honolulu, US： 2359–2367.
29.
LongJShelhamerEDarrellT (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of 2015 IEEE conference on computer vision and pattern recognition, Boston, US：3431–3440.
30.
Loukaitou-SiderisA (2005) Is it safe to walk here? Research on Women’s Issues Transp 102: 102–112.
31.
LuoJWuGWeiZ, et al. (2019) Developing an aerial-image-based approach for creating digital sidewalk inventories. Transportation Research Record: Journal of the Transportation Research Board2673: 499–507.
32.
MattyusGWangSFidlerS, et al. (2016) HD Maps: Fine-grained road segmentation by parsing ground and aerial images. In: Proceedings of 2016 IEEE conference on computer vision and pattern recognition, Caesars Papace, US: 3611–3619.
33.
NingHHuangXLiZ, et al. (2020a) Detecting new building construction in urban areas based on images of small unmanned aerial system. Papers in Applied Geography6: 56–71.
34.
NingHLiZWangC, et al. (2020b) Choosing an appropriate training set size when using existing data to train neural networks for land cover segmentation. Artificial Neural Network GIS:1–14.
35.
OsamaASayedT (2017) Evaluating the impact of connectivity, continuity, and topography of sidewalk network on pedestrian safety. Accident; Analysis and Prevention107: 117–125.
36.
PouryaS, ZareapoorM, ZhouH, et al. (2020) Road Segmentation for Remote Sensing Images Using Adversarial Spatial Pyramid Networks. IEEE Transactions on Geoscience and Remote Sensing1–16. https://doi.org/10.1109/TGRS.2020.3016086
37.
PrathapGAfanasyevI (2018) Deep learning approach for building detection in satellite multispectral imagery. In: Proceedings of2018 International conference on intelligent systems (IS). IEEE, Madeira Island, Portugal:461–465.
38.
RonnebergerOFischerPBroxT (2015) U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of 2015International conference on medical image computing and computer-assisted intervention, Munich, Germany: 234–241. Springer.
39.
SenletTElgammalA (2012a) Segmentation of occluded sidewalks in satellite images. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), Tsukuba, Japan: 805–808.
40.
SenletTElgammalA (2012b) Satellite image based precise robot localization on sidewalks. In: Proceedings of2012 IEEE international conference on robotics and automation, St Paul, US: 2647–2653. IEEE.
41.
ShiQLiuXLiX (2017) Road detection from remote sensing images by generative adversarial networks. IEEE Access6: 25486–25494.
42.
SmithVMalikJCullerD (2013) Classification of sidewalks in street view images. In: Proceedings of 2013 international green computing conference, Arlington, US: 1–6.
43.
UlkuIAkagunduzE (2019) A survey on deep learning-based architectures for semantic segmentation on 2D images. ArXiv191210230 Cs.
44.
VakalopoulouMKarantzalosKKomodakisN, et al. (2015) Building detection in very high resolution multispectral data with deep learning features. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS), Milan, Italy: 1873–1876.
45.
WangZLiuBSchulterS, et al. (2019) A parametric top-view representation of complex road scenes. In: Proceedings of 2019 IEEE conference on computer vision and pattern recognition, Long Beach, US: 10325–10333
ZhangXJiZZhengY, et al. (2020) Evaluating the effect of city lock-down on controlling COVID-19 propagation through deep learning and network science models. Cities 107. https://doi.org/10.1016/j.cities.2020.102869.
48.
ZhaoHShiJQiX, et al. (2017) Pyramid scene parsing network. In Proceedings of 2017 IEEE conference on computer vision and pattern recognition, Honolulu, US: 2881–2890
49.
ZhengCWangL (2014) Semantic segmentation of remote sensing imagery using object-based Markov random field model with regional penalties. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing8: 1924–1935.
50.
ZhouBLapedrizaAKhoslaA, et al. (2017a) Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence40: 1452–1464.
51.
ZhouBZhaoHPuigX, et al. (2017b) Scene parsing through ade20k dataset. In: Proceedings of 2017 IEEE conference on computer vision and pattern recognition, Honolulu, US: 633–641.
52.
ZhuXXTuiaDMouL, et al. (2017) Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine5: 8–36.
53.
Zuniga-TeranAAOrrBJGimblettRH, et al. (2017) Designing healthy communities: Testing the walkability model. Frontiers of Architectural Research6(1): 63–73.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
